Learning outcomes: The long goodbye: Vocational qualifications in the 21st century
Introduction: Why this topic?
The phrase ‘learning outcomes’ has, in the last few decades, become ingrained in the experience of almost everyone in the English-speaking world who either works in or experiences education. Yet one can ask educational professionals what a learning outcome is and receive different and often inconsistent replies. Although the terminology of learning outcomes is ambient, our understanding is limited. Perhaps learning outcomes is a family resemblance concept (Wittgenstein, 1953) whose ramified usage is so much part of the educational landscape that we need not worry about precise definitions. However the phrase conceals at least two approaches to education fundamentally at odds with each other and so long as this ambiguity remains, much damage will continue to be done, particularly to vocational education.
The language of learning outcomes and associated terminology such as ‘competence’, ‘output’ and ‘assessment criterion’ is becoming a global phenomenon, extending beyond the Anglosphere into Europe and the Global South (Allais, 2015). More than three decades after its adoption, its fortunes still seem to be increasing, despite a dubious record of achievement and some hard-hitting but relatively ineffectual criticism. It is difficult to find a clear answer as to why.1 As the basis for policy, the adoption of learning outcomes approaches has ranged from relatively harmless to disastrous2 and take-up has generally occurred without careful and thorough evaluation of previous experiences by early adopters. Of course, even if it ‘worked there’, it does not follow that it will ‘work here’ (Cartwright and Hardie, 2012), but finding out whether it ‘worked there’ is a sensible preliminary to any assessment of whether it will ‘work here’. Yet this is rarely done and is a testament to deep-seated problems in educational policymaking where reforms are too often undertaken without some basic precautions such as looking at previous experience and whether the approach considered is actually coherent, let alone looking at trade-offs between advantages and disadvantages of take-up (Joergensen, 2018; Lassnigg, 2012). A complete account of international take-up of learning outcomes approaches is beyond the scope of this article (but see Allais, 2015 for a relatively comprehensive account; Brockmann et al., 2011 discuss in detail European attempts to harmonise the European labour market through a learning outcomes approach).3 Instead, I will look at the experience of the UK and how that continues to affect European educational policymaking (see also Mikulec, 2017 for a useful account).
Learning outcomes and the rise of the NVQ
Although the ideas underpinning learning outcomes date back at least to the work of Rousseau and other progressive educational thinkers (Darling, 1993), Frederick Taylor and ‘scientific management’, behaviourist psychology (Allais, 2015; Hodge, 2007; Hyland, 1993), not to mention Public Choice Theory (Orchard and Stretton, 1994) and New Public Management (Ferlie, 2017), a synthesis coalesced in the UK between the mid-1970s and mid-1980s, starting with James Callaghan’s 1976 speech (Callaghan, 1976) on the quality of English education and precipitated by the rapid growth of youth unemployment from the mid-1970s onwards. Callaghan’s speech was also the start of a growing consensus about the need for more consistency in the English educational system, leading to the adoption of a national curriculum for England in 1988. These were really two distinct but related developments, but their co-occurrence led to a confusion still difficult to unpick in the present policymaking environment.
The incoming Thatcher administration of 1979 was determined to remove the corporatist vocational education and training (VET) structure underpinned by the 1964 Industrial Training Act, which led to the setting up of 27 industrial training boards (ITBs) (Pemberton, 2001; Raggatt and Williams, 1999). The intention was to move to an employer- and market-led VET system, leaving no more of the 1964 architecture than was strictly necessary. However, youth unemployment was an issue exacerbated by the recession of 1980–1981 that no government could afford to ignore. Thus was born the youth training scheme (YTS) and the new training initiative (NTI) in which the Manpower Services Commission (MSC) together with the Department for Employment were significant actors, while the Department of Education (DES) played a relatively minor role (Raggatt and Williams, 1999: 10–11, 80).
Callaghan was concerned with personal, civic and vocational preparation. He thought that education should ‘equip children to the best of their ability for a lively, constructive, place in society, and also to fit them to do a job of work. Not one or the other but both’ (Callaghan, 1976: 3). He advocated both a national curriculum and increased vocational relevance, pointing to the alleged inability of the education system to prepare young people for adult life where employment was the destiny of most. A VET system that appeared mired in antiquated practices such as time-served craft apprenticeships without an educational component and a jungle of hard-to-decipher qualifications was ill-suited to the demands of an economy where employers were dominant in the labour market and rapid technological and economic evolution was likely to render current know-how redundant (Raggatt and Williams, 1999: 20). There was a growing consensus among officials and politicians that an employer-led VET structure focused on competences needed in the workplace would be the key to equipping the economy for the demands of the late 20th century. The idea of competence became central to these efforts but, as will be seen, there is no tight link between the concept of competence and that of learning outcome (for more on this in the European context, see Sivesind and Wahlstrom, 2016). This consensus led to the consultations that eventuated in the National Vocational Qualifications (NVQs) in 1986. A consensus also emerged about the desirability of a national curriculum, also dating from Callaghan’s speech. These developments appeared closely related, but led to quite distinct results.
NVQs: Aspirations and reality
Let us look in more detail at the features of NVQs to identify the unrecognised and thus unresolved tensions between parties that led to the incoherence of the design. Qualifications that certify competence were considered important to the enhanced schemes designed to tackle youth unemployment – the YTS and the NTI – if they were to provide young people with a lasting labour market asset (Raggatt and Williams, 1999: 29–31). Employers’ representatives were concerned that any such qualifications should be a signal of ability to operate in the workplace and less about educational status. On the other hand, the officials and politicians designing the new qualification system were concerned that it should equip young people for the future as well as the present. There was an aspiration that NVQs would recognise higher-level abilities such as problem solving, planning, teamwork, etc. (Jessup, 1991: 26–30) and that they would come to dominate not just the technical, but the higher technical, technological and managerial regions of the labour market (Jessup, 1991: 26–29, Raggatt and Williams, 1999: 28–31). In the technique of functional analysis they thought that they had found a way of designing occupational profiles that began with the overall purpose of an occupation and then decomposed it into the specific abilities needed for workplace practice. In the intentions of the MSC at least, NVQs were a means of modernising and adapting the workforce, not just of certifying existing abilities. Already two concepts of competence were in contention and one of these won hands down.4
NVQs used the innovative concept of learning outcomes as a central design feature.5 A learning outcome in this sense is a statement of the abilities that someone should possess in order to obtain a qualification. It has a normative status even though it only describes such abilities. Typically, within the modular structure of most NVQ qualifications, there are three or four learning outcomes per module, each of which is further specified by a larger number of assessment criteria – more detailed specifications of what actions should be performed by the candidate to satisfy the requirement of a learning outcome. Satisfaction of all the assessment criteria is necessary and sufficient.
However, the new qualifications were introduced into an economy and labour market that were not always focused on high-level ability, autonomy and adaptability. The idea that Britain’s economy was a low-skill equilibrium emerged with Finegold and Soskice’s (1988) diagnosis of its modus operandi, dominated by low-skill, low-wage workers producing low-cost, low-specification products and services. Demand and supply are in equilibrium and there is a cost but no benefit to individual employers moving up the value chain, so they have little or no incentive to invest in VET. Exogenous supply of VET (e.g. by the state) will not necessarily result in increased demand by employers. More than 30 years on, Finegold and Soskice’s diagnosis has stood the test of time, albeit with qualifications about the regional and local rather than national location of skills equilibria (Sissons, 2020).
The realities of the English labour market led to the design and adoption of NVQs mainly at level 2, craft rather than technical labour, and ‘bottom-up’ job design, starting with the tasks to be carried out mapped onto the skills needed to perform them. This was the opposite procedure to that recommended by functional analysis but was to dominate qualification design at the lower and intermediate levels for the next 30 years. The conception of competence thus adopted pragmatically came to be that of the ability to carry out an array of workplace tasks (effectively defining an occupation) to a threshold quality level.6 The National Council for Vocational Qualifications (NCVQ) had no choice but to accept this reality if it wanted NVQs to gain widespread acceptance among employers (Raggatt and Williams, 1999: 98–103). It should also be noted that employers were usually not interested in what educational ‘level’ the qualification was deemed equivalent to, but rather whether it was a signal of competence.
On the other hand, officials were concerned with educational levels as well as workplace competence. NVQs needed to have dual value – on both the educational and labour markets – if they were to meet the needs of a highly skilled and adaptable workforce capable of continuing professional development. They were designed with the intention of showing an equivalence in educational value with existing academic qualifications, ranging from pre-GCSE to master’s degrees. Thus all NVQs were assigned a level indicating a rough equivalence with their academic counterparts. In fact, beyond level 3 (roughly A level equivalent), take-up was uncommon except in certain specialist areas, for reasons already indicated.
However, as NVQs were designed around a restricted conception of workplace competence, knowledge was not salient in their design. This feature – a consequence of their prioritisation of workplace competence – led to some epistemological contortions by their designers and advocates. Knowledge was said to be implicit in performance (Jessup, 1991: 125) or the underpinning knowledge for a performance could be elicited through post hoc questioning (Jessup, 1991: 124). It was even suggested that theoretical knowledge underpinning qualifications at a higher level, such as a pharmacy degree, was not strictly necessary for expert practice (Jessup, 1991: 126).7 As we shall see, none of these manoeuvres work if vocational qualifications are guarantees of professional competence at anything other than the most elementary levels of professional practice. How to conceptualise the place of knowledge and its relationship with ability has been an ongoing design problem for learning outcomes qualifications attempting to model complex professional abilities.
The broader educational vision outlined by Callaghan was thus lost in policymaking focused narrowly on employability. There was thus a disconnection between the discussions taking place on the national curriculum on the one hand and vocational education on the other, and in the latter case there was an unwillingness to engage with and understand key educational concepts such as the relationships between knowledge, curriculum and assessment. This lack of a shared vision affected not only different government departments, but also key stakeholders such as employers (Raggatt and Williams, 1999: 32–33, 70–74). A lesson from the NVQ design experience is that the involvement of all stakeholders and a common understanding of key concepts is required in successful qualification design. Later, we shall see whether that lesson was taken to heart by the EU and its policymaking bodies.
The fatal flaw: Progression and assessment
The placement of NVQs in levels of cognitive hierarchy arose from the need to match them to educational qualifications to facilitate educational progression. Employers attracted to NVQs valued them for their certification of workplace competence, not for their educational currency. But the fact that they were organised into a cognitive hierarchy also created the illusion that they were in some way comparable with more conventional educational qualifications and in particular with the newly emerging English National Curriculum (ENC). Jessup even asserted that the ENC was an outcomes-based structure (Jessup, 1991: 79). This confusion arose from a misinterpretation that has had fateful consequences for the development of outcomes-based qualifications.
Although the architects of the ENC believed it would ultimately benefit the economy, unlike NVQs it was not designed as a vocational instrument. Quite the contrary; it bore the hallmark of a liberal curriculum perhaps even modelled on Harold Wilson’s ideal of comprehensive schooling as ‘grammar schools for all’. However, the architects were concerned to make clear what pupils were meant to learn and at what age, so that progress could be assessed and stakeholders (parents and employers) could understand easily what was expected of pupils and teachers. Thus the ENC contained not only programmes of study (outlines of content) but also attainment targets (ATs) (Graham and Tytler 1993: 150–152). These latter were summary statements of the content that should be acquired in each of the curriculum subjects at the ages of 7, 11, 14 and 16. Grammatically they look superficially like NVQ statements of learning outcomes. Normatively, like NVQ learning outcomes, their achievement is necessary if progression is to be allowed. However, they have quite different conceptual properties from learning outcomes (LOs) in the NVQ framework.8
First, they are summaries of curricular content, not statements of workplace competence. They cannot be used independently of the content they summarise. LOs are to be understood only in terms of workplace competence. In the ENC mastery of content is assessed. Second, ATs are arranged explicitly in a cognitive hierarchy and are to be assessed successively. The attainment of targets at key stage 3 depends on the successful achievement of targets at stages 1 and 2.9 In other words, content at earlier stages has to be mastered before progression is allowed to higher levels. This requirement was contrary to that of NVQs which could be achieved independently of any prior attainments. Finally, ATs were not themselves assessment instruments but served as indicators for formal or informal, formative or summative assessments. LOs on the other hand were to be assessed through a decompositional procedure, whereby each LO was disaggregated into a number of assessment criteria (ACs), each of which had to be satisfied before the candidate achieved the LO.
Attainment targets and learning outcomes thus had different conceptual properties reflecting their different aims. ATs were linked to content and progression, LOs not. An AT could be assessed in a variety of different ways by a range of possible instruments. Each instrument was sufficient to determine whether the AT had been achieved by a pupil. Thus AT1 could be assessed by Instrument 1, 2 or 3, each of which on its own was sufficient for achievement of the AT, in the same way that different examinations in different years are deemed sufficient for award of a qualification as they each represent different samples of content. However an LO could only be reached if each of its assessment criteria were met. Thus LO1 was reached if and only if AC1 and AC2 and AC3 etc. were satisfied.10 Achievement of all the requisite ACs was both necessary and sufficient for achievement of the LO and achievement of all the LOs in a qualification was necessary and sufficient for award of an NVQ. It was and continues to be insufficiently appreciated how different were these requirements and how different content-based ‘learning outcomes’ are from LOs conceived in the NVQ mode. I will call the former approach, exemplified in attainment targets, a standards-based approach. The approach exemplified in the NVQ design I will call a learning outcomes approach.
In fact, the latter, in its attempted combination of independence and cognitive hierarchy, contains a flaw not present in ATs and other such summaries of curriculum content.
Formally:
-
1] LO1 of an NVQ at level 1 is to be awarded if and only if ACs 1, 2 and 3 are satisfied by the candidate.
-
2] LO2 of an NVQ at level 2 is to be awarded if and only if ACs 4, 5 and 6 are satisfied by the candidate.
-
3] The achievement of LO1 is necessary to the achievement of LO2. (This is the cognitive hierarchy requirement, that some abilities presuppose the prior possession of other, more basic, ones.)
-
4] Achievement of ACs 1, 2 and 3 is necessary for the achievement of LO2 (follows from 1] and 3]).
-
5] But this contradicts the requirement for LO2 set out in 2] above.
It is thus not possible to combine the cognitive hierarchy requirement, necessary if the NVQ is to be regarded as an educational qualification, with the independence requirement desired by employers interested only in workplace task fulfilment (Winch, 2015: 176). Even the independence requirement at any level other than the most basic will only work for types of employment that do not require nested abilities or extensive knowledge and are not safety-critical. This last stipulation is necessary since where an occupation is safety-critical, the possession of no important ability or knowledge can be left to chance or to non-explicit checking.
It is easy to see why some employers were not interested with cognitive hierarchy in 3] since they are concerned above all with in situ workplace competence (Raggatt and Williams, 1999: 36–37, 55, 70–74). However, a professional qualification is a social guarantee of professional competence for all stakeholders and cannot ignore the cognitive hierarchy requirement of 3]. Thus practice has been to focus on satisfying assessment criteria 2] alone for the award of NVQs above level 1 and 2 (MSC/DES, 1986: 26–27, cited in Raggatt and Williams, 1999: 65); thus they fail to provide a guarantee of integrated competence required by genuine professional qualifications. This might not matter too much in elementary and non-safety-critical occupations, but is an irredeemable flaw in professional qualification design more generally.11
There is some evidence that this fundamental flaw in the NVQ design was remarked on in the early years of its adoption (Raggatt and Williams, 1999: 26, 73–74; but see a contrasting attitude in Jessup, 1991: Ch.18), but it was never remedied.. Still less was an LO approach questioned, since it was identified with a broad shift in qualification design that went far beyond the NVQ phenomenon (Hodge, 2007). To this day the difference between standards and outcomes approaches is incompletely understood (Brockmann et al., 2008). We do know, however, that problems with NVQs emerged shortly after their adoption. They tended to be designed in a ‘bottom-up’ fashion (Raggatt and Williams, 1999: 71) . Occupations were constructed from tasks matched one-to-one with assessment criteria and learning outcomes, while the functional analysis approach recommended by Jessup was not followed. NVQs thus tended to predominate at levels 1 to 3 and their adoption at level 3+ was patchy to non-existent. This tendency reflected dominance of some influential employers in design and adoption and provided an early lesson in the ways in which powerful stakeholders can influence the trajectory of policy adoption (Raggatt and Williams, 1999: 70–71).12
The NVQ approach to occupational knowledge, treating it as implicitly manifested in workplace practice, is not viable. Assessors can only gauge underpinning and/or theoretical knowledge in an indirect and piecemeal fashion, leaving the danger of large gaps in professional knowledge that would only come to light outside the assessment context (Prais, 1991; Winch, 2016). This problem was recognised with the introduction of modern apprenticeships (MA) at level 3 from 1994 onwards, containing an NVQ for the practical elements, a technical certificate for the theoretical element and a functional skills element for basic literacy and numeracy. Discrete assessment of the different elements meant that they could not be treated as integrated in professional action. The need to incorporate a functional skills element in the MA, even at level 3, testified to the failure of the workplace learning approach to embed some of the key abilities necessary to perform in a satisfactory way (‘ the standard required to successfully perform an activity or function’; Jessup, 1991: 25), despite the claims of NVQ activists that interest in performance would lead to enhanced learning (Jessup, 1991: 4–5).
It is not surprising that in many sectors of the labour market, NVQs were considered to be low-level qualifications enjoying little prestige among either employees, prospective employees or many employers. Their adoption by 2006 was uneven, many employers knew little or nothing about them (Roe et al., 2006) and they attracted very poor or even negative returns in employment (Department of Business, Innovation and Science, 2011). They had failed in the marketplace competition in which it was expected that they would sweep away all rivals. This happened even in an economy largely configured as an LSE. Jessup’s claim that learners would be the primary beneficiaries of NVQs and of learning outcomes approaches more generally has turned out to be unfounded, yet this claim continues to be repeated within the European Union (Cedefop, 2017).
There is no large-scale evaluation of the NVQ innovation which was ready to hand for policymakers from other countries. That does not, however, absolve them from evaluating it carefully before adopting some of its key features. What lessons could policymakers in other jurisdictions have learned from the NVQ experience in England? I suggest the following:
-
1] England failed to evaluate the internal coherence of the NVQ proposal and in particular did not examine the logic of NVQ design in comparison with other kinds of qualifications, including those that bore a superficial similarity to the learning outcomes approach adopted by NVQs.
-
2] England failed to pilot NVQs before full implementation to ascertain any hidden design flaws and to assess whether or not the influence of the most powerful stakeholder (in this case certain employers) was likely to undermine the aims of the NVQ innovation.
-
3] England failed to carry out a cost-benefit analysis to determine whether the purported advantages of NVQs exceeded their disadvantages that their introduction. This could have been done through the implementation of pilots [2].
-
4] England failed to combine stakeholder consultation with truly independent evaluation of the coherence and practicality of the NVQ idea. Consensus does not guarantee that the idea agreed on is coherent.
Examining the coherence of a proposal, piloting it and carrying out a cost-benefit analysis (1] –3] above) are hardly policymaking ‘rocket science’, yet it is surprising how infrequently these precepts are followed (see Joergensen, 2018 for some Scandinavian examples). 4] is perhaps less obvious, both for systems dependent on one or more dominant stakeholders (e.g. state and employers) but also for more evenly balanced social partnership approaches where extensive consultation and balancing of interests is the favoured modus operandi.13
Careful study of the English experience would have prevented implementation of a flawed approach to qualification design. Far too few jurisdictions have bothered to do this. The Netherlands was initially attracted to the NVQ idea, although policymakers there failed to appreciate how radically detached from content and providers its designers intended it to be (Westerhuis, 2011: 72–73). However, the narrow task-based approach and lack of flexibility led to disillusion and the idea was dropped in favour of a more expansive conception of competence more akin to that of Germany (Westerhuis, 2011: 73–76). In this case, a consensus against an NVQ approach emerged. More recently the government of Ontario has taken extensive advice on previous experience with learning outcomes approaches before determining whether this should be the dominant principle in curriculum design, paying attention to point 4] (Arnold et al., 2021).
Developments in Europe 1: The European Qualification Framework (EQF)
Since the 1980s, the EU has considered economic development to be one of the aims of educational policy (Cino-Pagliarello, 2020). However, education and skills development is not one of its competences; this rests with national governments. It is possible, though, for the EU to initiate developments with national consequences through the open method of consultation (OMC), whereby different member states agree on approaches and ways of implementing them. This has been the approach adopted in educational and skills development policy. The European Qualification Framework (EQF), launched in 2008, initiated a reform process in member states by providing an impetus to either construct or recast national qualification frameworks (NQFs) in terms of learning outcomes. The EQF has no legal or normative force, it is a meta-framework or ‘translation device’ (Sellin, 2008) for comparing qualifications and qualification frameworks. This was to be done by recasting qualification frameworks (and eventually qualifications) in a learning outcomes framework. This itself was an approach recommended by the OECD to promote greater transparency and rationalisation of qualifications. It does, however, have a certain moral and persuasive force. Member states agreeing to its intentions are under a certain obligation to fulfil them. The main ones were: first, to enable comparison of qualifications within states in terms of progression along pathways from one qualification to another; and second, to enable the comparison of qualifications in different states with one another. Implicitly the EQF was also intended to promote labour mobility within and between member states by making comparison easier (Méhaut and Winch, 2012: 373). In terms of national adoption of NQFs nominally following the learning outcomes design philosophy, the EQF has been a success. Whether it has been successful in promoting qualification redesign and labour mobility is another matter. Ultimately the NVQs failed on the labour market through a lack of enthusiasm on the part of learners and employers. It is important to consider whether the EQF and other European policy tools based on learning outcomes will suffer a similar fate.
What does it mean to say that the EQF is based on a learning outcomes approach? The EQF, as stated, has no legal or normative force, it operates through persuasion (Méhaut and Winch, 2012: 369). It is a set of general descriptions of the kinds of knowledge, know-how and autonomy/competence to be expected of qualifications at each of its hierarchically arranged eight levels. The intention is that these descriptors are suitable for the formulation of NQFs and ultimately qualifications themselves in terms of learning outcomes.14 At this point it becomes more difficult to make sense of the learning outcomes approach in the European context, particularly because the EQF has no normative force. Two countervailing pressures are evident. From the EU and its agencies such as Cedefop, and also from some stakeholders such as some employer associations, features to be found in the NVQ development can be clearly discerned. First, there is a frequently stated intention to promote the interests of learners and to challenge providers of qualifications (Cedefop, 2017: 25–27) with the promotion of non-formal pathways towards achieving qualifications associated with informal and non-formal learning. This approach in turn makes the independence criterion for learning outcomes important. One should be able to achieve a qualification, at whatever level, independently of the achievement of a prior qualification at a lower level, as with LOs. However, since the EQF is a set of related descriptors, it cannot mandate any particular design philosophy among the qualifications that it is designed to enable comparison of.
However, the clearly stated intention to promote ‘learning outcomes’-based qualification design in the member countries makes it reasonable to suggest that the EQF was – at least originally – a transformative framework in the sense of Allais et al. (2009), designed to leverage radical qualification redesign in member countries. On the other hand, EU policymaking has to be consensual. Without the agreement of all the member states, adoption of a policy will not be possible. Although prior to the adoption of the EQF a broad consensus emerged that a learning outcome approach was desirable, it was and remains far from clear that each member state (or the stakeholders within each state) understood the same thing by ‘learning outcomes approach’ (Mikulec, 2017: 460). Some are happy with the NVQ approach; others – and these include some of the more powerful states within the EU, together with powerful stakeholders within these states – are much more comfortable with an approach that makes learning outcome descriptors, whether at EQF or NQF level, much more akin to the attainment targets of the ENC, in our terminology conceiving of the descriptors as standards rather than learning outcomes. They are thus associated at the national level with curricula and reflect those.15 The EQF is thus an umbrella for NQFs and national qualifications embodying more than one design philosophy. As pointed out, standards and learning outcomes approaches cannot be incorporated in the same qualification. One approach requires content and a non-independent approach to achievement of the qualification, the other specifically mandates that content need not be specified and that the award can be made independently of the acquisition of other qualifications.
This incompatibility may not matter at the national level. It would, however, have serious consequences for cross-national labour market adoption, since it cannot be assumed that qualification Q1 in Country A, nominally equivalent in EQF level to Q2 in Country B, is really equivalent, even if they both use a ‘learning outcomes’ framework as descriptors. If Q1 is designed on a standards basis and Q2 on an LO basis, crucial attributes such as conception of competence, underpinning knowledge, contextual factors and assessment regime may not be sufficiently clear to effect a reliable comparison for labour market purposes without a great deal of extra information. The EQF could then be no more than a starting point for more detailed templates for the direct comparison of qualifications (Brockmann et al., 2011: 1–21). And, given that there is no one coherent conception of learning outcomes common to all member states, and that, indeed, at least two conceptions are incompatible, it may be time to reconsider whether it is helpful to call the EQF a learning outcomes-based framework. The current handbook (Cedefop, 2017) emphasises the advantages for learners of a learning outcomes approach in terms of clarity and the ability to take qualifications independently of others. But as the NVQ experience suggests, independence of itself is of no advantage if the qualification itself is of poor quality and reputation. As the EQF is essentially a descriptive rather than a normative framework, there is no reason why it should not be detached from learning outcomes.
The EQF has the potential, with a great deal of work, to be adapted as a labour market tool. It can thus develop more detailed descriptors for sectoral purposes (Garstka and Syben, 2009) and for scope of activities (Brockmann et al., 2010, for construction occupations; Galla et al., 2014 for furniture making). But it can serve this purpose whether or not the qualifications or frameworks to be compared are designed to any kind of outcomes approach or none. It is not a normative framework and the descriptors can be fitted to a variety of different qualification designs. Learning outcomes are an idle wheel in the machinery, the main purpose of which is the comparison of qualification frameworks and qualifications which they can only do if the underlying design and assumptions of the qualifications to be compared are understood.
Developments in Europe 2: European Skills, Competences and Occupations
Since 2013 a new VET policy tool has been under development and trial. This is European Skills, Competences, Qualifications and Occupations (ESCO), originating in the European Directorate responsible for employment and developed independently of the EQF. The core of ESCO is a skill inventory comprising 13,485 skills and 2,942 occupations (European Commission, 2018) that can be updated according to labour market needs. The design of ESCO is based on the principle used in the International Standard Classification of Occupations (ISCO) of a digital classification to associate high level sectoral groups (1 digit), through generally described occupations (2 digits), occupations (3 digits), ending with quite detailed occupational descriptions (4 digits), ESCO taking the classification further with more detailed descriptors than ISCO’s and then specifying the skills required at the lowest level of the hierarchy (European Commission, 2019: 16–17). Originally there was no hierarchy within the skills classification. Knowledge requirements for occupations are subsumed within the skill classifications (European Commission, 2019: 21–22). ESCO skill occupation descriptors are not framed as learning outcomes; each skill can be mapped onto a learning outcome, so that an occupational profile consisting of many skills can eventually be matched with a qualification embodying that mapping.
Originally ESCO did something that some employers wanted NVQs to do, but was resisted by those fearful of their losing their educational currency: namely, locate all skills at the same level without any attempt at creating a cognitive hierarchy.16 Now, although it is claimed that there is a hierarchy of more or less general attributes, the Skills Pillar displays this hierarchy at the most detailed level:
This allows bringing transversal knowledge, skills and competences which are rather abstract to a more detailed level so that they can be directly used in occupational profiles. (European Commission, 2018: 20)
There seems to be no reason why ESCO-generated occupational profiles cannot be mapped onto a variety of differently designed qualifications, both LO- and non-LO-based. However, the designers’ idea seems to have been that one could map each ESCO skill onto a corresponding learning outcome. It would then seem as if a qualification corresponding to a profile could be constructed without reference to any other qualification or without assigning the qualification thus constructed to any particular place in a cognitive hierarchy of level in a qualification framework. The original radical learning outcome approach foreshadowed in the development discussions for the NVQ would be realised. But, as we have seen, it is not possible, if a qualification is to have educational currency and allow for progression in learning, for it not to be situated in an educational hierarchy, which itself presupposes a cognitive hierarchy. Most knowledge presupposes the possession of other knowledge, the possession of know-how that of the possession of other know-how (Winch, 2016). Very often knowledge is organised in terms of complexity and abstractness, while know-how is also organised in terms of complexity, difficulty and a foundation of pre-existing ability, the possession of some skills presupposing others. The skill of writing with a pencil presupposes fine motor manual skills. In addition, there is a transversal aspect to know-how. Some forms of know-how can be realised through different skill sets and cannot be identified through descriptions of the outcomes of the exercise of discrete skills, but rather in the overall purpose of the activity to which the ability is applied (Ryle, 1979). Communication with a business partner may be effected through speaking or writing, for which particular skills are required. These skills may neither be necessary (because others could be used) nor sufficient (because the writer’s focus was on the exercise of the skills rather than the communicative aim).
We should also distinguish between transversal abilities in the sense just described and transferable abilities applicable across a range of activities (Winch, 2015, 2016). In general, transversal abilities have limited transferability. Planning a year’s lessons is a different ability from planning the construction of a railway bridge. Transferable abilities tend to be skills rather than transversal abilities. The ability to write in one’s own hand is realisable in different contexts and the success criterion is the production of text. Contrast the ability to write with the ability to communicate, which can be realised in different ways and the object of which is to get an interlocutor to understand what is communicated.17
The attempt to differentiate the Skills Pillar hierarchically began in 2018 with an adaptation of the Canadian National Skills Classification, a two-level structure distinguishing skills and knowledge intended to be independent of occupational groups (European Commission, 2018: 3). By contrast, the ESCO hierarchical classification, based on the 13,485 items in the Skills Pillar, adds another two levels so that the hierarchy moves from very general descriptions of skills and knowledge to ones that fit into an occupational classification. In addition, the ESCO classification also distinguishes, within the Skills Pillar, transversal abilities and attitudes and values. Does this approach address the problems of hierarchy and progression that accompanied the NVQs from their outset?
To resolve this, some clarification is necessary. First, contrary to what is implied, one cannot assume that there are very general forms of know-how – for example, there is no second-level skill of ‘social interaction’ or ability such as ‘thinking’.18 Such categories can serve as placeholders for identifiable items of knowledge, skill, competence and attitudes and values, but they do not designate properties of an individual. The ESCO hierarchy can organise the selection of items in the Skills Pillar into occupational profiles of the kind familiar to qualification designers. These second-level categories are also used to designate elements in some national qualification frameworks. They bear more than a passing resemblance to the kinds of standards-based descriptions that can be found in the occupational profiles for the English Apprenticeships introduced in the last few years. Such occupational profiling has no privileged relationship with learning outcomes in the radical (LO) sense, since ESCO descriptors can serve as standards indicating elements of knowledge, etc., that the curriculum for an occupational qualification must address through assessment. In short, the ESCO hierarchical classification of the items in the Skills Pillar can be used to construct occupational profiles of which the actual place in a cognitive/educational hierarchy and the design principles are left open.
Policy tools, qualifications and national conceptions of competence
Earlier it was observed that the EQF, in order to have labour market traction, needed its categories enriched and content provided. ESCO, on the other hand, appears to start with a level of detail very close to the needs of the labour market. There is little evidence that either has achieved this traction.19 ESCO can be used to create something like an occupational profile, with some level of detail as to skills, knowledge, transversal abilities and attitudes provided from a selection of the 13,485 items in the Skills Pillar. There is no a priori answer why ESCO has apparently had little labour market impact, but there are some conceptual grounds for thinking that this should not be surprising.
Professional qualifications are a social guarantee of professional competence. The assessment of a candidate for eligibility for award of a qualification is the means whereby this guarantee is maintained. It is vital for the integrity of the occupation and the confidence of clients and the public in the ability of members of the occupation to carry out their role. Qualifications have to respect the needs of various stakeholders: learners, employees, employers, the occupation, government, trade unions, clients, the public, and providers of the qualification (Winch, 2016: 555–556). They cannot therefore privilege the interests of one group to the detriment of the interests of others, although some qualifications may favour one of these groups. If they do so too much, they risk losing the trust of other stakeholders. NVQs were designed with the needs of learners and employers predominantly in mind, and the interests of providers were deliberately downgraded. In practice, the interests of some employers, mainly those interested in certificating semi-skilled labour, predominated and the system failed to gain confidence among broader sections of stakeholders, including learners. The learning outcomes discourse underpinning the European VET policy tools employs a similar rhetoric and appears, likewise, to favour learners as stakeholders.
Unlike NVQs however, there is no dominant discourse concerning learning outcomes among the different national stakeholders and it is possible to espouse different and contradictory conceptions while using the same language. It is thus possible, for example, for German trade unions to support the EQF structure as it can be used to provide equivalence at level 6 of Bachelor degrees with the Meister certificate for senior and experienced workers. But a structure founded on incompatible presuppositions is not secure. Professional qualifications are a guarantee of competence, but different national conceptions of professional competence are reflected in their design (Brockmann et al., 2011: 9; Telling and Serapioni, 2019). For the NVQ, although a broad and flexible, if nebulous, conception of competence was originally envisaged, in practice the view of an influential group of employers – that competence should be the threshold performance of bundles of narrowly conceived tasks – came to dominate.20 The EU is too diverse for this to happen.
ESCO is the nearest it has come to designing an instrument capable of direct application in the labour market but it has not, and could not have, any official conception of competence underlying it. Its original non-hierarchical and fragmentary Skills Pillar looked as if it was designed with something like an NVQ approach to determining employment readiness in mind, but in practice it has not been used for this purpose. Given that different countries employ different conceptions of competence in their qualification design and labour markets, this should not be surprising. There is no European conception of competence, but rather families of countries with more or less similar conceptions, generating a degree of mutual trust in the probity and usefulness of each other’s qualifications (Brockmann et al., 2011). Neither is there a European conception of learning outcomes. Different stakeholders in different countries have different and often incompatible conceptions. The EQF and ESCO are not directly tied to any conception of learning outcomes, they can be used with or without any such conception. Their role in comparing nominally equivalent qualifications in different countries is therefore limited so long as that role is not accompanied by an examination of the different national conceptions of competence in play.
The demise of the NVQ in England
The shortcomings of NVQs have been rehearsed. Criticism had been taking place for a long time, but the Wolf Report (Wolf, 2011) was particularly influential, leading to a change in attitude towards qualifications by government. This move was reinforced by the Richard Review of apprenticeships (Richard, 2012) and the Whitehead Review of qualifications (Whitehead, 2013). The non-learning outcomes-based Apprenticeship Standards and the introduction of the Regulated Qualification Framework (RQF) in 2015 (Ofqual, 2016) signalled the demise of the design principles underlying NVQs. In 1986 NVQs were proclaimed to be a revolutionary development designed to replace existing qualifications. By 2015 their design principles were discarded. Within 30 years a revolution and a counter-revolution in the design of British vocational qualifications took place. However, although in many ways the critique of LO-based qualifications was acute and to the point, all three of these reports avoided making a direct assault on the principles underlying NVQs. There is a danger that, in failing to get to the root of the problem with the NVQ conception of learning outcomes, mistakes made in the past will be made in the future, as indeed they are in Europe and many other parts of the world.
A theme that runs through all three reports is that too many qualifications are based on narrow and over-prescriptive occupational standards and are thus inflexible.21 Qualifications written in this way are too rigid for occupations to keep up to date with new developments (Richard, 2012: 7; Whitehead, 2013: 18; Wolf, 2011: 88). Wolf also notes that the LO approach can drive down standards of performance. In the terms this article has set out the problem, if one LO is fulfilled if and only if six assessment criteria are met, then the qualification can only be met if 100% success on the assessment criteria is achieved. This leads to a temptation to lower pass thresholds or to simplify assessment criteria so that a sufficient number of passes can be achieved. By contrast, standards-based approaches to assessment are not so rigid yet also ensure that the overall standard for the qualification is met. Wolf also notes that the attitudes of some employers can also provide the impetus for narrow and fragmented qualifications, referring to ‘employer demand for a specific sub-set of our many qualifications and indifference to others’ (Wolf, 2011: 69). Thus all three reports argue for a standards-based approach whereby general aims in terms of knowledge and know-how are specified, sufficiently flexible to take account of new developments and capable of different forms of assessment suited to changing and particular needs. In the terms of this article, they recommend a standards-based rather than an LO approach to qualification design. Wolf gives an example of the extremely rigid manner in which the Qualifications and Curriculum Authority (QCF) prescribed the writing of LOs and comments:
Then, as now, this exercise did virtually nothing for the substantive quality and uniformity of the actual assessment (or training) process, where empirical studies confirm that NVQ assessment standards are highly variable. (2011: 101)
The point is thus made that, for all the rhetoric about the benefits for learners of an LO approach, if anything the quality and hence the benefits for learners of such qualifications are negligible without other elements of quality. As Wolf points out:
Such outcomes are inevitable in a system which regulates on the basis of written specifications, because a written specification plays only a small part in determining what is actually taught, let alone the standard and quality of the assessment. (2011: 101, bold text author’s own)
Wolf is very clear that vocational education should assume and build on a sound basis of general education and that the requirements of initial vocational education (IVET) are distinct from those of continuing vocational education (CVET). The LO approach, in emphasising the ‘work readiness’ of the candidate through matching work tasks to assessment criteria, blurs this vital distinction and thus contributes to the low status and labour market orientation of LO-based qualifications. Not surprisingly, this low status is reflected in some negative labour market returns, particularly for levels 1 and 2 of the NVQ (Department of Business, Innovation and Science, 2011: 151). Even if NVQs were to be of some use in certifying low-level competences in some occupations, this would be a bad reason for using them for IVET purposes.
Wolf, Richard and Whitehead make little or nothing of the fact that the LO approach has difficulty in conceptualising the integration of different attributes such as systematic knowledge and attitude in professional action. But if the conception of competence with which one is working cannot encompass the integration of both systematic and proximal knowledge in professional judgement and action, it is hardly surprising that the award of NVQs was largely confined to low-level occupations requiring largely routine tasks.
A decent burial – but first, an inquest
One must conclude that the introduction of NVQs and of the LO approach to qualification design more generally is one of the outstanding disasters in English educational and labour market policymaking in recent decades. The lives of many people persuaded to acquire qualifications of little or no value have been damaged and the economy has suffered from a dearth of properly qualified employees, while the problems of the low skill equilibrium have persisted. The Wolf, Richard and Whitehead reports recognise this disaster, but their brief was to provide remedies, not to inquire into the causes. In order to prevent such disasters reoccurring in England and other parts of the world, an investigation of both the design flaws and their conceptual underpinning together with the policymaking process that allowed such a disaster to happen was needed.
Instead, the NVQ (but not its name) has been quietly interred deep in the pages of the Regulated Qualification Framework (RQF; Ofqual, 2016). From page 58 onwards we learn about the requirements for guided learning as a proportion of total qualification time (TQT) required for an award, and it is clear that guided learning consists of both formal (classroom) learning and informal (e.g. supervised work practice and formative assessment) rather than informal learning (e.g. unsupervised practice). Assessment is broadly framed so that it is fit for purpose rather than complying with an LO-based procedure: ‘the rationale for the proposed assessment framework, and the extent to which it would facilitate the valid assessment of the required knowledge and skills’ (Ofqual, 2016: 61) is thus presented as one of the indicators that a qualification is likely to conform to the RQF.
Taken together, these requirements reflect the recommendations of the three reports discussed above and effectively seal the demise of regulated LO-based qualifications in England, the main virtue of which was said to rest on the possibility of acquiring them through informal learning. However, the sub rosa manner in which these changes are presented and the fact that some qualifications will still be entitled to call themselves NVQs suggests that the government is not anxious to publicise the magnitude of the change. However, England provides an invaluable case study of a natural experiment in an LO-based approach in a large country, driven for some years with great determination, but which ultimately failed. There can be no reason for countries seeking to introduce learning outcomes into their education systems, at whatever level, not to study the English experience carefully and to learn from the mistakes that were made, both in design and conception.
Whither Europe and learning outcomes?
This article has told the story of how a flawed policy innovation in England was adopted, without proper evaluation, as a template for Europe-wide reform of vocational qualifications. The evidence suggests that there have been both ideological and pragmatic elements to these reforms.22 The ideological element relates to the elevation of the interests of the learner and the element of challenge to providers. There are affinities with the NVQ development in England. There are grounds for thinking that educational reforms that ignore the quality of the teaching and learning that underpin qualifications are unlikely to succeed (Fullan, 2006). This lesson should have been learned from the NVQ experience. The pragmatic element relates both to the wish to cut through the complexity of comparing curricula and the wish to develop instruments for use in a European labour market. In addition, the achievement of consensus contributes to a false sense of security if other elements of sound policymaking are ignored. As is well known, the EU prides itself on its consensual approach to policymaking, both the EQF and ESCO being the outcomes of such processes, albeit given a strong steer by the Commission and the EU’s VET agency Cedefop. But consensus can form around a mistake. Ideology and over-hasty pragmatism can form a dangerous mix when further research, reflection and caution are needed instead. If care is not taken to learn from past mistakes, such as the NVQ experience, consensus building will not guarantee quality of qualifications but reinforce the original mistake. Caution is also needed as the actors in Europe’s many labour markets often have divergent interests. One cannot reasonably expect widespread enthusiastic local labour market take-up from a Europe-wide innovation without careful preparation. Above all, though, study of and reflection on past relevant experience is the first necessary, but not sufficient, step towards sound policymaking. The EU claims that it does not wish to emulate the NVQ, but unless it gets to the bottom of its design flaws and detaches the learning outcome approach from its policy tools, it will continue to struggle. One can add that any broad educational vision of the kind invoked by Callaghan has been lost in these EU initiatives in favour of a utilitarian approach that places little or no emphasis on civic and personal dimensions of VET. So what should be done?
First, there is too much of a fixation on the language through which qualifications are described. The key to comparing and assessing qualifications from different countries is the underlying concept of competence that each embodies (Brockman et al., 2011; Telling and Serapioni, 2019). Framing qualification descriptors in terms of learning outcomes is ill-suited to capture such often holistic conceptions. Both the EQF and ESCO are compatible with descriptors that do not involve learning outcomes either of the broad-based standards variety or the narrow NVQ-like ones. The surface grammar of descriptors is a poor guide to their significance and content. These must be related to context and underlying conception of competence to make sense. However, the ‘bare bones’ approach of the EQF can be developed into a much more detailed framework in which different conceptions of competence embodied in different qualifications can be mapped and compared.23 Another missing link for the EQF – the description of the contents of qualifications – can be partly remedied through the Skills Pillar of ESCO. Admittedly, the categories of such frameworks and their content descriptors need to be contextualised in order for the comparison to make sense (European Commission, 2019: 21), but a significant stride could be made towards comparison and to provide a conceptual map of competence for curriculum designers (Winch, 2015). But all this can be done without recourse to either the vocabulary or the concepts of learning outcomes – a term that has no definite or coherent meaning. The EU should do what the English have failed to do and analyse what is wrong with learning outcomes so that progress can be made. Meaningful labour market-relevant comparison of VET qualifications is possible (see the discussion in Brockmann et al., 2011: Ch. 9) and both EQF and ESCO can contribute to that aim, but will be better off doing so without insisting on learning outcomes.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
ORCID iD
Christopher Winch 
https://orcid.org/0000-0003-0818-5102
Notes
1.
But, according to Allais (2015: 34), there has been growing disillusion outside Europe, e.g. South Africa. However, a neighbouring country, Namibia, is currently adopting this approach.
2.
For the former, the use of the term to mean the aim of a programme or subprogramme characterised by a curriculum, in the case of some European vocational qualifications or the attainment targets of the English National Curriculum. The latter includes the reorganisation of qualifications in South Africa (Allais, 2015) and the NVQ development in England (see below).
3.
Brockmann et al. (2011) suggest that this can be done without the use of learning outcomes.
4.
See Telling and Serapioni (2019) on competing conceptions of competence at the European level.
5.
The central idea of competence-based training (CBT), on which learning outcomes are based, had its origins in attempts to improve training in the US armed forces on the one hand and to reform teacher education in the United States on the other. It was heavily influenced by behaviouristic psychology (Hodge, 2007: 196–201). The reform of VET in England was the first large-scale national attempt to implement CBT (Hodge, 2007: 205–206).
6.
But see also the ESCO Handbook for a similar conception of the aggregation of skills to form occupations (European Commission, 2019: 22).
7.
This claim results from a misreading of Boreham’s (1989) study of the work of pharmacists. In fact Boreham is at pains to stress the integration of academic with local knowledge, together with the situational awareness of the pharmacists themselves.
8.
The acronym LO will be used to refer to the NVQ conception of learning outcomes.
9.
There was a tension between the idea that Key Stages were age- and cognition-related. But there was not the political appetite for holding back pupils until they had fully met the attainment targets of each Key Stage.
10.
See, for example, City and Guilds (2015).
11.
If the relevant level 1 qualification is taken prior to the level 2 one, there is no problem of omission. But to do this is to discard the independence requirement considered to be such an advantage of NVQs.
12.
Trade unions tended to acquiesce in this approach from employers. A notable exception was the electricians’ union, EETPU, which devised qualifications that differed significantly in their design from the NVQ approach and attracted a great deal of electrician employer interest.
13.
A large part of the problem in implementing the EQF was tight timescales imposed by the EU, which prevented a logical unfolding from qualification to qualification framework (Lassnigg, 2012: 300–301), although two promoters of the EQF argue that time limitation was a virtue in this instance (Bjørnåvold and Coles, 2007–2008: 228).
14.
NQFs may themselves be purely descriptive. They may have normative force, as has the English Regulated Qualification Framework (RQF), by making certain properties mandatory if a qualification is to achieve national recognition, or they may have legal force if all qualifications must conform to the stipulations of the NQF. The EQF does not mandate any particular kind of NQF.
15.
This does not mean that they exclude validation of prior learning (VPL) procedures, which take account of experiential and informal learning.
16.
The generic term ‘skills pillar’ now includes skills, competences, knowledge and attitudes.
17.
Even transferable abilities such as writing become more specialised as one links them to more specific activities, usually located higher in educational hierarchies and thus much less transferable.
18.
It could be argued, however, that attitudes and values do not have this level of specificity, although the generality of virtues is a matter for philosophical debate (Hursthouse and Pettigrew, 2016: 1.1). For a critique of the concept of highly general abilities, see Röeben, 2018: 386–388.
19.
According to ESCO, no more than 39 private organisations have made use of the tool. See: https://ec.europa.eu/esco/portal/howtouse/bfe2a816-f9dd-49df-a7d2-ec8fafcfce95
The situation looks no better for the EQF; the EU emphasises its influence on national policy, not on labour market use: https://www.cedefop.europa.eu/en/events-and-projects/projects/european-qualifications-framework-eqf
(both accessed 25 July 2020).
20.
It is associated with a narrow conception of an occupation which persists in the post-NVQ era.
21.
Examples include NVQs at levels 1 and 2. The City and Guilds Level 2 Diploma in Health and Social Care (City and Guilds, 2015) provides an example and the associated assessment criteria are, in effect, even more detailed and prescriptive learning outcomes.
22.
See Jessup, 1991 for the more ideologically based arguments; Raggatt and Williams, 1999 for a careful account of the pragmatic considerations that weighed with policymakers and officials.
23.
For an example, see Clarke et al., 2020: 2.
References
|
Allais, SM (2015) Selling Out Education: National Qualification Frameworks and the Neglect of Knowledge. Rotterdam: Sense. Google Scholar |
|
|
Allais, SM, Raffe, D, Young, M (2009) Researching NQFs: Some conceptual issues. ILO Employment Sector Employment Working Paper No. 44. Google Scholar |
|
|
Arnold, C, Wilson, M, Bridge, J, Lennon, MC (eds) (2021) Learning Outcomes, Academic Credit, and Student Mobility. Montreal and Kingston: Queen’s University McGill-Queen’s University Press. Google Scholar |
|
|
Bjørnåvold, J, Coles, M (2007–2008) Governing education and training: The case of qualifications frameworks. European Journal of Vocational Training 42/43 double issue: 2007 (3) 2008 (1): 203–235. http://www.cedefop.europa.eu/EN/Files/42-en.pdf. Google Scholar |
|
|
Boreham, NC (1989) Modelling medical decision-making under uncertainty. British Journal of Educational Psychology 59(3): 187–199. Google Scholar | Crossref | Medline |
|
|
Brockmann, M, Clarke, L, Winch, C (2008) Can performance-related outcomes have standards? Journal of European Industrial Training 32(2/3): 99–113. Google Scholar | Crossref |
|
|
Brockmann, M, Clarke, L, Winch, C (eds) (2010) Bricklaying is more than Flemish bond. Brussels: CLR. https://www.academia.edu/10176692/Bricklaying_is_more_than_Flemish_bond_Bricklaying_qualifications_in_Europe. Google Scholar |
|
|
Brockmann, M, Clarke, L, Winch, C (eds) (2011) Knowledge, Skills and Competence in the European Labour Market. Abingdon: Routledge. Google Scholar | Crossref |
|
|
Callaghan, J (1976) A rational debate based on the facts. Speech given at Ruskin College, Oxford, 18 October 1976. Google Scholar |
|
|
Cartwright, N, Hardie, J (2012) Evidence-based Policy: A Practical Guide to Doing it Better. Oxford: Oxford University Press. Google Scholar | Crossref |
|
|
Cedefop (2017) Defining, Writing and Applying Learning Outcomes: A European Handbook. Luxembourg: Publications Office of the European Union. DOI:10.2801/566770 Google Scholar | Crossref |
|
|
Cino-Pagliarello, M (2020) Aligning policy ideas and power: The roots of the competitiveness frame in European education policy. Comparative Education. https://www.tandfonline.com/eprint/QN5YTZSSBSDGFGVRVIGY/full?target=10.1080/03050068.2020.1769927#metrics-content Google Scholar |
|
|
City and Guilds (2015) Level 2 Diploma in Heath and Social Care (Adults), England. London: City and Guilds. Google Scholar |
|
|
Clarke, L, Sahin-Dikmen, M, Winch, C (2020) Overcoming diverse approaches to vocational education and training to combat climate change – the case of low energy construction in Europe. Oxford Review of Education 46(5): 619–636. Google Scholar | Crossref |
|
|
Darling, J (1993) Rousseau as progressive instrumentalist. Journal of Philosophy of Education 27(1): 27–38. Google Scholar | Crossref |
|
|
Department of Business, Innovation and Science (2011) Returns to Intermediate and Low Level Vocational Qualifications. BIS Research Paper No. 53. London: HMSO. Google Scholar |
|
|
European Commission (EC) (2018) Proposal for Implementing a Skills Hierarchy in ESCO. Brussels: European Commission. Google Scholar |
|
|
European Commission (EC) (2019) ESCO Handbook: European Skills, Competences, Qualifications and Occupations. Brussels: European Commission. DOI:10.2767/934956 Google Scholar | Crossref |
|
|
Ferlie, E (2017) The new public management and public management studies. Oxford Research Encyclopedias, Business and Management. DOI:10.1093/acrefore/9780190224851.013.12 Google Scholar | Crossref |
|
|
Finegold, D, Soskice, D (1988) The failure of training in Britain: Analysis and prescription. Oxford Review of Economic Policy 4(3): 21–53. Google Scholar | Crossref |
|
|
Fullan, M (2006) Change theory: A force for school improvement. Centre for Strategic Education Seminar Series Paper No. 157. Australia: Victoria. Google Scholar |
|
|
Galla, B, Gehring, R, Leijtens, J, Truong, N, et al. (2014) Handbook for European Furniture Professions. Brussels: EFBWW archive. http://www.efbww.org/pdfs/Bolster%20up%20Handbook%20EN.pdf. Accessed 27 August 2019. Google Scholar |
|
|
Garstka, B, Syben, G (2009) Sectoral Qualifications for the Construction Industry in Europe. Bremen: BAQ Forschungsinstitut. Google Scholar |
|
|
Graham, D, Tytler, D (1993) A Lesson for Us All: The Making of the National Curriculum. London: Routledge. Google Scholar |
|
|
Hodge, S (2007) The origins of competency based training. Australian Journal of Adult Learning 47(2): 179–209. Google Scholar |
|
|
Hursthouse, R, Pettigrew, G (2016) Virtue ethics. Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/entries/ethics-virtue/#Aca Google Scholar |
|
|
Hyland, T (1993) Competence, knowledge and education. Journal of Philosophy of Education 27(1): 57–68. Google Scholar | Crossref |
|
|
Jessup, G (1991) Outcomes: The Emerging Model of Education and Training. London: Taylor and Francis. Google Scholar |
|
|
Joergensen, C (2018) A problem-oriented approach to comparison of VET in the Nordic countries: Exploring common trade-offs for policy. Presentation at the 3rd International G.R.E.A.T. Conference, Cologne, Germany, September 2018. Google Scholar |
|
|
Lassnigg, L (2012) ‘Lost in translation’: Learning outcomes and the governance of education. Journal of Education and Work 25(3): 299–330. Google Scholar | Crossref |
|
|
Manpower Services Commission/Department of Education and Science (MSC/DES) (1986) Review of Vocational Qualifications in England and Wales. London: HMSO. Google Scholar |
|
|
Méhaut, P, Winch, C (2012) The European Qualification Framework: Skills, competences or knowledge? European Educational Research Journal 11(3): 369–381. Google Scholar | SAGE Journals |
|
|
Mikulec, B (2017) Impact of the Europeanisation of education: Qualifications frameworks in Europe. European Educational Research Journal 16(4): 455–473. Google Scholar | SAGE Journals | ISI |
|
|
Ofqual (2016) Guidance to the General Conditions of Recognition. London: HMSO. Google Scholar |
|
|
Orchard, L, Stretton, H (1994) Public Goods, Public Choice and Public Enterprise. London: Macmillan. Google Scholar |
|
|
Pemberton, H. (2001) The 1964 Industrial Training Act: A failed revolution. Paper presented in the new researchers section of the conference of the Economic History Society, Bristol, UK, 30 March 2001. Google Scholar |
|
|
Prais, SJ (1991) Vocational qualifications in Britain and Europe: Theory and practice. National Institute Economic Review 136: 86–89. Google Scholar | SAGE Journals |
|
|
Raggatt, P, Williams, S (1999) Government, Markets and Vocational Qualifications: An Anatomy of Policy. London: Falmer. Google Scholar |
|
|
Richard, D (2012) The Richard Review of Apprenticeships. London: Department of Business, Innovation and Science. Google Scholar |
|
|
Roe, P, Wiseman, J., Costello, M (2006) Perceptions and use of NVQs: A survey of employers in England. BMG Research. file:///C:/Users/zszz4812/OneDrive/Chris%20stick%20backup%2014.4.19/Research/projects/2018/Cedefop%20compare/WA3/Perceptions%20and%20Use%20of%20NVQs%20A%20Survey%20of%20Employers%20in%20England.pdf Google Scholar |
|
|
Röeben, P (2018) Kompetenz und Expertiseforschung. In: Rauner, F, Grollmann, P (eds) Handbuch Berufbildungsforschung (3rd edition). Bielefeld: Media Gmbh, 384–392. Google Scholar |
|
|
Ryle, G (1979) On Thinking. Oxford: Blackwell. Google Scholar |
|
|
Sellin, B (2008) The proposal for a European Qualifications Framework. Making it a reality – possibilities and limitations. European Journal of Vocational Training 42(3): 4–18. Google Scholar |
|
|
Sissons, P (2020) The local low skills equilibrium: Moving from concept to policy utility. Urban Studies. DOI: 10.1177/0042098020915859 Google Scholar | SAGE Journals |
|
|
Sivesind, K, Wahlstrom, N (2016) Curriculum on the European policy agenda: Global transitions and learning outcomes from transnational and national points of view. European Educational Research Journal 15(3): 271–278. Google Scholar | SAGE Journals | ISI |
|
|
Telling, K, Serapioni, M (2019) The rise and change of the competence strategy: Reflections on twenty-five years of skills policies in the EU. European Educational Research Journal 18(4): 387–406. Google Scholar | SAGE Journals | ISI |
|
|
Westerhuis, A (2011) The meaning of competence. In: Brockmann, M, Clarke, L, Winch, C (2011) Knowledge, Skills and Competence in the European Labour Market. Abingdon: Routledge, 68–84. Google Scholar |
|
|
Whitehead, N (2013) Review of Adult Vocational Qualifications in England. London: UKCES. Google Scholar |
|
|
Winch, C (2015) Towards a transparency tool for professional and vocational qualifications. Journal of Education and Work 28(2): 165–186. Google Scholar | Crossref |
|
|
Winch, C (2016) Assessing professional know-how. Journal of Philosophy of Education 50(4): 554–572. Google Scholar | Crossref |
|
|
Wittgenstein, L (1953) Philosophical Investigations. Oxford: Blackwell. Google Scholar |
|
|
Wolf, A (2011) Review of Vocational Education – The Wolf Report. London: HMSO. Google Scholar |
Christopher Winch is a Professor of Educational Philosophy and Policy in the School of Education, Communication and Society, King’s College London. His main interests are in Philosophy of Education and Comparative Vocational Education.
