A total of 304 training courses delivered by external training companies were examined and 32,700 assessments, representing 4,825 learners, were reviewed. Whilst this review was taking place, FÁS suspended the process of requesting certificates from awarding bodies in order to ensure that all training is properly assessed and thereafter certified. Following the review process, assessment outcomes for over 95% of the learners were deemed valid.
A number of non conformances were highlighted by this thorough exercise. The non conformances ranged from less serious, non-deliberate to serious non conformances. The review found:
• 261 courses (86%) were in compliance. 43 courses (15%) required rescoring of assessments, re-sits, or re-training and re-sits due to the nature of the non-conformances. Of these, 3 courses (1%) represented serious non conformances which involved the deliberate manipulation of results or inappropriate assistance to candidates
• The 43 courses represented 718 learners or 15% of the total reviewed. Provisional results stood for 526 (11%) of these learners, thus resulting in 192 (4%) learners having their grades affected as follows:
- 116 learners had their grade revised downwards (2.4%)
- 22 learners had their grade revised upwards (0.5%)
- 40 learners on four courses are awaiting results of their re-sits (0.8%) and 14 learners on one course are awaiting adjudication of their provisional results (0.3%).
FÁS has taken immediate steps to correct problems which arose and to introduce new measures in order to tighten assessment procedures on its training provision including:
• A new system to upgrade training standards is being rolled out with an emphasis on more robust and frequent monitoring of FÁS training and assessment systems.
• New contracted training procedures and revised contracts including more robust sanctions for non-compliance have been developed and are due to commence this month.
• A revised tender framework with the aim of raising the quality of the training offered to FÁS clients is being introduced.
• A new assessment marking scheme will be introduced in the first half of 2011 which will enhance the alignment of FÁS' marking and grading criteria with FETAC requirements.
FÁS Director General, Paul O’Toole wished to re-assure FÁS learners ‘while the review highlights areas requiring a need for considerable improvements , there is a relatively small number of cases with serious issues which FÁS is working to resolve. Other learners should be assured of the quality of their certification. All assessments are examined and certified in accordance with industry standards. We apologise to any affected learners and commit to the improvement plans detailed in this review’.
Speaking on the publication of the review Chairman, Michael Dempsey said: ‘We are very conscious of the need to provide a flexible service that meets the needs of the labour market in the current difficult conditions. FÁS is committed to improving its systems to ensure it delivers the highest possible quality service to all its learners. Corrective action has been taken and learners can be assured of the highest possible standards’.
A copy of the Report is available to download here:ReportContractedTrainingOct4th.doc
For further information contact: Maria Walshe, FÁS Communications Unit, 27-33 Upper Baggot St, Dublin 4; Tel: 01 6070514, Mob: 087 2074280, Email: email@example.com Notes to editors
A total of 304 courses were subject to a three phased examination.
Phase 1 consisted of a monitoring visit by competent FÁS staff to each of the courses. The main purpose of the visit was to ascertain how well the course was being conducted when measured against FÁS requirements for contracted training courses. This Phase concluded and the initial outcomes reviewed by FÁS Senior Management in January 2010.
Phase 2 consisted of a review of assessment practices by competent FÁS staff pertaining to each of the courses monitored during Phase 1. In this regard, and where available, a sample of assessments relating to each course were reviewed. Over 32,000 assessments were reviewed. The main purpose of Phase 2 was to form a view as to whether or not the integrity of the assessment and certification system was breached and where a potential breach was identified, to immediately initiate an in-depth examination of the course. Phase 2 concluded and the initial outcomes reviewed by FÁS Senior Management in June 2010.
Phase 3 consisted of an independent ratification of the work conducted by FÁS during Phases 1 and 2. This ratification exercise was conducted by WRC Social and Economic Consultants. This phase was completed at the end of July 2010. See Appendix 1 for the full WRC Report (in the review).
In addition to, but outside of the scope of the review of contracted training, other assessment and certification issues identified within the organisation and through customer complaints were collated on an on-going FÁS register. Details of these issues and the corrective action taken are also included in Part 3 of this report. Scale of non-conformances
A Full Compliance - no evidence of non-conformances in any of the documentation reviewed or in any of the comments and observations of the FÁS personnel consulted during Phase 3.
B Assessment Procedure Related Non-Conformances 1 - mainly concern non-conformances in respect of the use of Form F11 and non-matching of the actual dates on which assessment events took place with dates specified on F11.
Example: Date that assessment took place is not the same as the date on the Form F11 notifying in advance the date of the test.
BB Assessment Procedure Related Non-Conformances 2 - concern issues such as the use of inappropriate tests or significant but non-deliberate breaches of assessment procedures.
Example: A paper had been modified (questions and answers), used as a mock paper and then original paper used for the assessment. Trainees re—sat a different test.
C Assessment Marking, Grading and or Evidence Related Non-Conformances 1 - include inconsistencies with model answers, presence of errors in marking, incorrect or incomplete Summary Assessment Sheets, errors in transferring marks to Summary Assessment Sheets, incorrect totalling of marks, errors in transcription of marks or results, incorrect grades being awarded to candidates, missing assessment evidence.
Example: Clerical Errors such as exams marked correctly but results not inserted into relevant documentation. Results from individual learner assessment sheet incorrectly transcribed onto the class Results Summary Assessment Sheets.
CC Assessment Marking, Grading and or Evidence Related Non-Conformances 2 - same as the above but with a higher prevalence of non-conformances or more serious but non-deliberate non-conformances in the courses falling into this category.
Example: ‘Essential’ areas of competence required not clearly visible on marking sheet and not applied by the marker.
D Presence of Both Categories B and C Non-Conformances - courses in which non-conformances in respect of both Categories B and C as described above were found.
E Course Content, Assessment Tests, and Marking Schemes Related Issues - used to categorise courses where the main outcomes and observations of reviewers concerned course content or design (e.g., out-dated tests, need for revision of module content, inaccuracies in model answers, need to review marking systems or instructions).
Example: Out of date model answers in the Sales Techniques Assessment have been updated.
F Serious Non-Conformances - as per definition provided to reviewers (i.e. manipulation of assessment data, submitting false assessment evidence, submitting falsified results, providing learners with access to a controlled assessment instrument prior to the assessment taking place, inappropriate handling of the assessment event).
Example: Intentional awarding of marks not achieved by learner.