Practice\hawkins

Introduction
For nearly two years now, an important part of the RSS Centre’s work has been that associated with the DfEE-funded HE Discipline Network project entitled MeaNs (Matching Education, Assessment and Employment Needs in Statistics). This project, based on a core partnership of Nottingham, Nottingham Trent, Sheffield and Sheffield Hallam Universities, has been managed throughout by Peter Holmes. It is concerned with the undergraduate training of specialist statisticians, and of those whose work involves statistics or working with statisticians. The ultimate aim has been to establish a national network of trainers, employers and employees, committed on an on-going basis to finding ways of enhancing the employability of graduates.
back to top

It is clear that, given its modest level of funding, the MeaNs project had to focus on information-gathering and dissemination rather than engaging actively in implementing change. However, the MeaNs project is now approaching its last month of funding, and it is time to consider the ‘ends’ so far achieved by the project team, and to identify ‘means’ by which to continue the work.

While any mismatch between training and employment needs is only apocryphal, it is difficult to use it as an influence for change. Factual evidence is needed, and the MeaNs project has used various sources of information in order to investigate whether there is a real mismatch and, if so, what its nature is. The earliest information came in response to announcements promoting the aims of the project. Peter Holmes spoke at ASLU-96 about some of the expressions of interest that had been received by the project team, and the experiences that were described by contributors (Holmes, 1996). Further notes are also available on the project web-site [http://www.maths.nott.ac.uk../] and in the first MeaNs newsletter.

Once the project was up and running, four workshops were held, bringing together employers, academics and recent graduates. These took place in Nottingham, Woking (at SPSS UK Ltd), London (at the RSS), and Glasgow. They provided the opportunity for further information gathering, and for the exchange of experiences and dissemination of earlier input, as did a series of visits to employers.
back to top

The issue of transferable skills has been a recurring theme, and one that has been echoed in the Dearing Report (1997). The DfEE has produced a number of publications on the subject, and other HE Discipline Networks are also concluding that there may be a mismatch between what tends to result from more traditional forms of university education and the transferable skills that are thought to be necessary for employment. The former may of course emphasise a certain type of ‘transferable skill’, but it tends to be one that is firmly rooted within the subject matter of the discipline itself, and that is consequently less generalizeable in different work contexts.

The questions that confront us, however, are; Can we identify the relevant transferable skills? What is their nature? What distinction should we draw between skills and knowledge or understanding? And what can we do to encourage the desired skills to emerge, assuming that we have correctly identified them? At present, it appears to be open season for producing mission statements about the aims and objectives of education. It is less easy to find examples of demonstrably ‘better’ pedagogy in action. I use ‘better’ with considerable caution here, because the project team has detected a certain amount of circularity between some of the statements about desirable outcomes and desirable teaching strategies.

In a recent paper concerning legal education, Duncan (1997) asked whether transferable skills are those ‘which can be presented at such a basic level and in such a general way that they appear to be relevant to any future activity of the learner? Is it those skills which are inherently relevant in a variety of contexts?’ He concluded, somewhat cynically, that he suspected that transferable skills really comprise any skills that fall outside the context of legal practice!

The members of the DfEE Discipline Network in Law, headed by John Bell at Leeds University, have drawn a distinction between the six core transferable skills that they think are appropriate to legal study and those that they feel relate to the concept of ‘graduateness’.

Six core skills	‘Graduateness’
Professional responsibility	Knowledge & understanding
Teamwork	Analysis
Communication	Synthesis
Problem solving	Evaluation
Personal skills	Creativity
Intellectual skills	Presentation

I do not see anything in these lists that we in the MeaNs project have not heard being described as desirable outcomes of undergraduate statistical, as opposed to legal, education. At the same time, however, we are conscious of variation in the extent to which institutions believe that they are already achieving these objectives, and in the extent to which they believe that change is necessary in their teaching approaches to achieve them. The project team has also observed differences in what steps towards change are perceived to be relevant, and in what steps institutions are able and prepared to make. The question of who should have the responsibility for developing transferable skills can also prove difficult. There are certainly those lecturers who see their prime responsibilty being almost, if not entirely, restricted to the area of developing students’ intellectual skills (or maybe that of knowledge and understanding).

The empirical part of the MeaNs project has been centred on four investigations or surveys of; academics, graduate destinations, employers, and post-university ‘top’-up courses.
back to top

Of the 85 university statistics departments contacted, 36 responded. Of these 14 claimed that they develop their courses with general, often informal, consultation with employers. 15 said that they offer courses aimed at specific areas of employment, e.g. pharmaceutical applications (and often therefore emphasising SAS), medical and environmental sciences.

Industry placements were used fairly extensively by these respondents. They were appreciated both by the academics, because their students tended to return with better attitudes and motivation, and also by future employers who then found their graduate recruits to be more mature and better equipped for the work-place.

Employers were a regular source of data and provided real examples for use by these statistics departments. Nearly all the respondents used practical and/or project work, but problem-solving and other transferable skills were usually expected to emerge as a result of this, rather than being specifically taught. There were reports of specific transferable skills courses tending to fail, either because they were mis-timed (e.g. study skills courses which were run right at the beginning of year one, before the students could appreciate their value) or because the courses were not assessed, with the result that students saw no reason to attend them. There was also some evidence that, given the choice, students would avoid courses that emphasised transferable skills because they perceived such courses to be more demanding.

The assessment formats used by these statistics departments tended to be fairly conventional. They did not really reflect the wide range of possibilities now featuring in the research literature. Exercises of the type used later in assessing the students were rarely used as a teaching medium. For example, rather than teaching report-writing skills, academics usually assessed these (often by their absence) at the end of a course.

Clearly, there are some limitations on how the information from this enquiry should be received. Firstly, this was a survey of statistics departments. It did not include departments where statistics might be taught without recourse to members of the statistics department. Practices with respect to service teaching do seem to vary from institution to institution, but it is likely that non-specialist training courses were under-represented in the replies. Secondly, a response rate of less than 50% is worrying, because a bias towards more innovatory departments might be expected.
back to top

A database of 444 distinct vacancies was compiled between March 1996 and May 1997 from ALLSTAT, national newspapers and professional newsletters. The objective was to identify the sort of statistical work for which graduates are recruited, and what is required of these graduates once they are in post. Advertisements for higher grades were also included in order to establish an upper limit on the requirements and possibilities for recent graduates.

Using Qualitative Data Analysis techniques, 25 descriptive variables were identified and coded;

The posts spanned 11 main application areas. Entry-level posts in pharmaceutical, social/behavioural sciences, business or government contexts did tend to be open to first degree graduates. A postgraduate qualification (not necessarily a Doctorate), however, was generally required in order to enter employment in the medical, as distinct from biostatistics, field, and for advancement thereafter. Computational statistics and science/engineering contexts seemed to have two rather distinct levels of recruitment; one for candidates with a first degree only, and the other for candidates with a Doctorate.

The required qualifications were classified as ‘first degree’, ‘post-graduate (Diploma, Masters or Doctorate), and ‘Doctorate’. In fact, only one out of the 444 advertisements mentioned a Diploma as being a possible pre-requisite post-graduate qualification. Some advertisements stated that the posts were for ‘professionally qualified’ statisticians, but only three specifically referred to CStat (Chartered Statistician) status.

Four main types of experience were identified. The first one, ‘varied experience’ was noticeable by its absence. Only three of the advertisements mentioned this, although varied experience might well be a distinct advantage to statisticians needing to handle a varied consultancy load. In contrast, approximately one third of the advertisements specified experience with particular types of statistical procedures. A (different) third demanded IT or computing skills, or experience of using a particular piece of statistical software, e.g. SAS. Finally, a third of the advertisements asked for applicants with experience of the field of application. The tendency to supplement first degree courses with postgraduate experience (and qualifications) of a more applied kind is the conventional route to many employment opportunities. Expectations that this will be the case take the pressure off first degree courses to prepare students for employment. Rather, these tend to be seen as preparation for further, possibly more applied, training.

Members of the project team had expected that ‘good communication skills’ would feature prominently, because this was what employers and educators were telling them. In fact, relatively few of the advertisements specified this requirement. Failure to draw attention to such a key aspect of most jobs involving statistical duties may serve to de-emphasise the need for training in relevant skills, making it even less safe for advertisers to ‘take it as read’ that potential recruits will possess this quality.

There was a general under-emphasis of the required personal characteristics. Different vacancies, though, may well suit different people. For example, some posts need people who are good ‘parallel processors’, who can operate on several projects at once, prioritising their work to meet appropriate deadlines. Others suit ‘serial processors’ who prefer to concentrate on one task to its completion, and who do not cope well with the pressure of many competing deadlines. Both may be admirable employees, but not necessarily in the same work context. It might be expected that universities would have a part to play in helping students to identify and develop their personal work-styles, and yet there seemed to be no real incentive for them to do so based on what was said in the advertisements. It is clearly important for employers and employees to understand what is required for a particular post if the ‘square peg in a round hole’ syndrome is to be avoided.

At the very least, lack of detail in advertisements is an example of the under-use of the employer’s voice to communicate the nature of employment needs to the educators. It is also a missed opportunity for promoting statistics and the professional skills of statisticians. It certainly makes the recruitment process a great deal more hazardous for both employers and employees. Precise job specifications can provide telling insights into how universities can make their graduates more employable. For their part, universities would do well to take heed if they wish to continue to attract students, especially when those students will increasingly be looking for good returns for their tuition fees.
back to top

The DfEE research contract prevented the project team from conducting a large survey of employers. The team therefore decided to obtain responses from a wide range of different types of employers, and to avoid doubling up on those who had already had input to the MeaNs project in other ways. For example, some governmental agencies had been involved in the workshops and were not therefore included in this part of the project. The 21 employers who did take part in this ‘survey’ fell into three groups;

It was found that these employers appeared to vary in terms of what they defined as a ‘statistical employee’. Generalisations and comparisons across companies are therefore not easy to draw. Recruitment to specialist or elementary/general grades also differed.

Interviews were the main form of assessment strategy that these respondents used for evaluating potential recruits. Sometimes these were structured around the applicant dealing with a given (or chosen) practical problem. Many of the companies said that they required little specific content knowledge, but rather wanted their recruits to possess a thorough understanding of basic concepts of probability, statistics and inference. They did not necessarily share the same view of what constitutes ‘basic’, though. Drug companies, for example, tended to want a more theoretical framework than that which is required by some of the other employers. Some employers merely said that they required their applicants to have followed and understood the material in ‘a standard BSc’ - whatever that may be!

The survey schedule that was sent to the employers prompted them to consider which of the aptitudes and characteristics they wanted their recruits to possess. The prompts used were those that had emerged from the investigation of Graduate Destinations, see (b) above. Not surprisingly, therefore, there was a tendency for the respondents to say that they would like recruits to possess these same characteristics (even though these did not always accord with what they actually put in their own recruitment advertisements). Some new ideas did arise, however, and are shown below in italics. Most of the requirements mentioned proved to be rather general, as opposed to being specific to statistical work and to statisticians;

The survey also investigated in-house and continuing training opportunities. Somewhat in contrast to what the survey of statistics departments had indicated, companies rarely felt that they had influence on the content of university undergraduate courses However, they often used external university course providers to suit their particular in-house training purposes. The employers presumably felt that they were able to exert more influence at the continuing education level, rather than directly at the undergraduate level.
back to top

The content and approaches of post-university top-up courses (as opposed to postgraduate taught courses and research degrees) are barometers of employment needs. Courses and workshops only run where there is a demonstrated need, and people who are willing to pay to attend! The existence of post-university courses that can be successfully (and repeatedly) marketed by universities, commercial concerns and other organisations may therefore indicate ways in which undergraduate training is failing to satisfy graduates’ subsequent employment needs. Clearly, though, the statistical training of some undergraduates is more comprehensive than that of others, or will correspond more exactly to their subsequent employment needs, irrespective of whether the graduates are specialists in statistics or in other disciplines. The objectives, content and approaches of undergraduate courses are all subject to variability, as are workplace requirements.

Details of post-university top-up courses were collected over a period of fifteen months spanning 1996 to 1997. These top-up courses may supplement and/or complement undergraduate training. Their content can be classified into four categories;

Often the content and approaches of top-up courses are entirely appropriate for post-university continuing education. Training and development should not end with the first-degree certificate, and statistics itself is a developing subject area. However, it can be inferred that some of the approaches adopted in some undergraduate courses may not be appropriate. For example, a course that is too theoretical can make the content inaccessible to the students, or fail to provide the applied tools and skills needed for work contexts. Certainly, much of the material and/or approaches offered in top-up courses are apparently needed so frequently, or across so many areas of application, that they might more appropriately be incorporated into undergraduate training. For example, categorical data analysis is especially popular with medical researchers, but is also wanted by biostatisticians or epidemiologists and social scientists.

Some of the top-up training courses appear to be at a surprisingly low level (e.g. introductions to data analysis, or to survey methods, for social scientists) which may suggest that quite fundamental aspects of statistics are being omitted from undergraduate courses. Alternatively, the problem may be that the students are failing to absorb the material when they do encounter it in their first degree courses. Either explanation, however, has to be a cause for concern that should result in some rethinking of how statistics is being taught.

The possibility that the undergraduate provision might in some sense be inadequate also highlights the need for flexibility for changing and developing undergraduate courses so as to reflect or anticipate future statistical developments. Teachers should be willing to eliminate material that is redundant (e.g. old methodology) or inappropriate (e.g. for the needs and level of non-specialist students). In undergraduate courses, there does seem to be a tendency to try to freeze statistics into a kind of time warp, at an intermediate level of technology. For example, some academics would rather ban the use of calculators in examinations, ostensibly to avoid students having (unfair) access to different techniques, than rethink the possibilities that such technology offers for enhancing the assessment process and the nature of the statistics that can be taught. For most people, the use of technology is an integrated part of the practice of statistics. It should not therefore feature merely as an add-on module divorced from the rest of an undergraduate’s statistical education.
back to top

The MeaNs project team has produced a number of further reports on its findings. These can be viewed at the web-site. A final evaluation of the project will shortly be produced and made available there also. Clearly, the work of the MeaNs project falls within the general remit of the RSS Centre, which is to enhance statistical understanding and learning at all levels and within all contexts. Some continuation of the MeaNs project initiatives is therefore guaranteed, subject to resource limitations. Today, however, we should consider what else can be done to sustain the momentum of the project, and how.

The project team has heard of many interesting initiatives and innovations, both within academic institutions and also in employment contexts. These are part of the considerable progress in statistical education, both nationally and internationally, that has occurred over the past 25 years or so. There is still a long way to go, however. In particular, the broader issue of how to overcome the conservatism within Higher Education, and bring about change, remains to be resolved. Much has been written about better ways of teaching statistics, but it is not proving easy to implement these recommendations on a wide scale. Most of the innovations that the MeaNs project has observed and reported are relatively isolated and self-contained, although it is possible to identify some developmental trends that are emerging.
back to top

The problem of how to stimulate change in Higher Education is not a new one. It has, for example, been addressed by the Computers in Teaching Initiative (1997 a & b), the Centre for Research on Learning and Instruction (Entwistle et al, 1992), and is certainly about to feature prominently in TLTP Phase 3. The Dearing Report (1997) is obviously another potential source of pressure for change, although the government’s response to this is still awaited.

Figure 1 shows a variety of strategies that were used by different teams in the DfEE’s Enterprise in Higher Education initiative (Hawkins [not the present author] & Winter, 1997). All have their merits, and the MeaNs project has embraced several of them. It might be suggested that the ‘Pincer Movement’ is the one that seems to be most obviously applicable. This is not really the case, however, because as described in the figure, the ‘Pincer Movement’ strategy implies that academics would be passive recipients of the tactic. In the MeaNs project, they have been heavily involved. Indeed, it has been difficult to secure such widespread, or equivalent, involvement from employers.

THE HALF NELSON ‘You have signed a contract!’
THE VIRUS APPROACH Spreading the virus and hoping the institution does not become immune.	THE 0891 APPROACH I’ll be anything you like.	MASTERING THE RUBIC CUBE The art of selling benefits. EHE has an attractive face to present to everyone.	TABOO BUSTING Confronting the issues: ‘skills’ is not a dirty word.
CONSCIENCE TICKLING Appealing to teacher professionalism – ‘for the good of the students’.	THE PARTYGOER Becoming part of the club, making connections with senior decision-making groups	THE TROJAN HORSE Finding a way into departments and building on every available link.	BRUSHFIRE TACTICS One big bonfire can be extinguished. Light numerous tiny fires and let them spread.
	ZULU WARRIORS Sending in the suicide troops who cause a stir but inevitably perish in the battle.	THE PINCER MOVEMENT Using employers and students as joint levers.
Hawkins & Winter, 1997

Figure 2, again from Hawkins & Winter (1997), indicates the type of people that are to be found working in Higher Education (and probably in most other large organisations). It is these people who must be persuaded to implement change, or else to act as persuaders themselves. A real problem in the field of statistical education has been that for much of its history it has tended to be driven mostly by the concern of enthusiasts (shown on the bottom right of Figure 2). As the Enterprise in Higher Education report suggests, though, more progress might be made if its cause could be taken up by ‘credible champions’ (shown on the top right corner of Figure 2) who are ready, willing and able, and also in positions of power. Do we have the support of enough of these people? If not - why not?

Another vital question is - What is it about statistics and statistical education that prevents statistical reasoning from being identified as a ‘key life skill’? There seems to be widespread, if not universal, agreement among statisticians that statistical reasoning is an amalgam of those skills identified by DfEE (1996), Dearing (1997), etc., plus a few more that are more discipline specific. However, when key skills are discussed (together with the UK’s need for them, and the strategies, and more significantly the resources, for developing them), the emphasis is on numeracy, literacy, IT aptitude, and Communication. What do we have to do to get statistical reasoning on to this list? How can we persuade other people to recognise its role in contributing to the proper development of these other ‘key skills’? A project like MeaNs can do a number of things - raise awareness, point to what needs doing, suggest improvements, etc. It was, however, only modestly funded, so its ability to implement change is inevitably limited. What can be done about this?

The Centre for Research in Learning and Instruction (Entwistle et al, 1992) outlined the following influences on effective learning; Entry Qualifications & Previous Knowledge Syllabus Content Study Skills Teaching Methods Assessment of Students Evaluation of Courses

The issues that surround course evaluation are indeed complex. It is generally accepted that student evaluations can play a part in obtaining an overall picture, but they should not be relied on exclusively. There are too many hard-to-control variables to influence the outcome of such evaluations. Peer appraisal can be useful, but generally not unless the exercise is repeated sufficiently frequently, and not unless a climate that is receptive to change can be established. A good system would probably be one that looks at improving as well as at evaluating teaching (i.e. one that is formative as well as summative). To what degree are the Teaching Quality Assessment exercises going really to motivate objective review measures, however? After all, in whose interest is it to ask those evaluation questions that will result in unfavourable answers!

The findings of the MeaNs project certainly support the view that some of the above influences feature far more than others in our course planning. The ‘triumvirate’ of Content, Methods and Assessment often seem to prevail over all other considerations. Certainly Assessment appears increasingly to be the major factor for students, while Content and Methods often seem to be the principal concern of the teachers, given that what can be easily assessed will tend to be what is taught.

‘But I have to cover the syllabus content!’ How often do we hear this as a justification for perpetuating less than optimal pedagogy. I am reminded of the joke - a traveller asks a local resident for directions, only to be told that if the local was going there he wouldn’t be starting from here! Statistical education has evolved to the point where it is at present, but there is a case for saying that this is not the right starting point for where it should be going in the future. The question arises as to who was instrumental in putting the syllabus content there in the first place? At HE levels, it is usually the lecturers themselves, so why can they not choose to do things differently?

David Moore (1997) refers to the tendency for academics to be misled by the Professional’s Fallacy - This is what I was taught, and the way that I was taught it, so it must be the right (and by implication the only) way to proceed. Many of our students, though, are not ‘us’ a few years down the line. Statistics has changed, and so too has the pool of students, and their needs. As Entwistle et al (1992) state, planners must remember ‘the changing nature of knowledge, [and] the need for graduates to be able to assume responsibility for their own continuing education’.

Course content may sometimes constitute a form of academic protection, i.e. if the course content is not sufficiently difficult (obscure or inaccessible), then teachers feel that they cannot really be academically credible to their immediate colleagues, or to colleagues in other departments and other institutions. The prime function of teaching is communication, however. It is our job to find ways of teaching the principles or basics of our subject to the widest possible audience, not of limiting those who can avail themselves of this understanding. Not all students will proceed to become specialists in statistics but, without a shared understanding of the basics, how will specialists in statistics and in other areas communicate with each other to good effect? If our subject content is supposedly ‘credible’, and yet our students remain ‘in-(or non-)credible’, we are achieving nothing and, at the same time, we are damaging the image of the subject.

I would suggest that we rarely give instruction in common sense, nor do our teaching and assessment strategies generally convey the notion that common sense is a worthwhile goal of Higher Education. The expression ‘back to basics’ is misleading - ‘Forward to basics’ is preferable, and yet not all our colleagues see things this way. Lupton also cited another, earlier, writer (unfortunately unattributed) who argued that people need to learn about and use probability in scientific research, saying that ‘the more ignorant a man is, speaking generally, the more certain he is of the correctness of conclusions derived by invalid methods from incorrect premises’. While it is not surprising that people should have needed persuading of the fundamental importance of the skills associated with statistical and scientific reasoning in 1892 when Lupton was writing, why are we still having to say the same things today, more than 100 years later?

Complaints are often made that the current teaching methods do not produce students who understand, or have the skills to apply, what they have been taught. More constructivist skills-teaching approaches, however, are felt to be too time-consuming (and hard to assess). Furthermore, the required skills do not necessarily seem to emerge even when the students are exposed to such approaches, e.g. to practical projects. Most of these problems can be attributed at least in part to a lack of pedagogic training for HE lecturers, and indeed to the marginalisation of teaching skills. We may not have enough knowledge about how to teach more effectively, but the knowledge that we do have is not always implemented (and perhaps it cannot be) in the current political/academic climate.

On a personal level, I worry about the assumption that more and more networks, and networks of networks, and networks of networks of networks, …. with their increasingly complex language of acronyms, …. will solve the problem. I am waiting for ‘implosion day’! The Enterprise in Higher Education initiative led to the conclusion that, although one successful strategy for change was ‘letting a thousand flowers bloom’, ‘at the end of the day somebody has to shape the garden’! In the case of statistical education, there are already some very pretty flowers growing, as well as some healthy looking vegetables, but we still need to recruit an army of gardeners to nurture them and to pull out the weeds.
back to top

Computers in Teaching Initiative (1997a) Embedding Technology into Teaching: achieving Institutional Change. Active Learning, Number 6, July 1997

Computers in Teaching Initiative Support Service (1997b) IT & Dearing – the Implications for HE. Ed. H Beetham. ISBN: 0 9531683 0 1

Dearing Report (1997) Higher Education in the Learning Society. Report of the National Committee of Enquiry into Higher Education, chaired by Sir Ron Dearing. ISBN 1-85838 254-8

DfEE & the Cabinet Office (1996) Competitiveness Occasional Paper - The Skills Audit: A report from an Interdepartmental Group.

Duncan Nigel (1997) The Skill of Learning: Implications of the ACLEC First Report for Teaching Skills on Undergraduate Law Courses. Web Journal of Current Legal Issues, 1997 Issue 5, http://www.ncl.ac.uk/~nlawwww/1997/issue5/duncan5.html

Entwistle Noel, Sheila Thompson & Hilary Tait (1992) Guidelines for Promoting Effective Learning in Higher Education. Centre for Research into Education and Instruction

Hawkins Peter & Jonathan Winter (1997) Mastering Change – Learning the Lessons of the Enterprise in Higher Education Initiative. Department for Education and Employment.

Holmes Peter (1996) Matching Education, Assessment and Employment needs in Statistics: The MeaNs project. Association of Statistics Lecturers in Universities, 1996 Proceedings. Printed in Teaching Statistics. ASLU Supplement 1997. Pp 1-3

Lupton Sydney (1892) On the educational value of the theory of probability. The Journal of Education, No.276, page 357-9

Moore David (1997) New Pedagogy and New Content: The Case of Statistics. International Statistical Review, 65: 123-65
back to top

RSS Centre for Statistical Education
The University of Nottingham
Nottingham NG7 2RD
Phone: +44 (0) 115 951 4911
Fax: +44 (0) 115 951 4951

Please email rsscse@ntu.ac.uk with any comments or corrections.

RSS Centre for Statistical Education The University of Nottingham Nottingham NG7 2RD Phone: +44 (0) 115 951 4911 Fax: +44 (0) 115 951 4951

Please email rsscse@ntu.ac.uk with any comments or corrections.

RSS Centre for Statistical Education
The University of Nottingham
Nottingham NG7 2RD
Phone: +44 (0) 115 951 4911
Fax: +44 (0) 115 951 4951