Section 658.601 - State Workforce Agency responsibility(a) Each SWA must establish and maintain a self-appraisal system for ES operations to determine success in reaching goals and to correct deficiencies in performance. The self-appraisal system must include numerical (quantitative) appraisal and non-numerical (qualitative) appraisal.(1) Numerical appraisal at the ES office level must be conducted as follows:(i) Performance must be measured on a quarterly-basis against planned service levels as stated in the Unified or Combined State Plan ("State Plan"). The State Plan must be consistent with numerical goals contained in ES office plans.(ii) To appraise numerical activities/indicators, actual results as shown on the Department's ETA Form 9172, or any successor report required by the Department must be compared to planned levels. Differences between achievement and plan levels must be identified.(iii) When the numerical appraisal of required activities/indicators identifies significant differences from planned levels, additional analysis must be conducted to isolate possible contributing factors. This data analysis must include, as appropriate, comparisons to past performance, attainment of State Plan goals and consideration of pertinent non-numerical factors.(iv) Results of ES office numerical reviews must be documented and significant deficiencies identified. A corrective action plan as described in paragraph (a)(6) of this section must be developed to address these deficiencies.(v) The result of ES office appraisal, including corrective action plans, must be communicated in writing to the next higher level of authority for review. This review must cover adequacy of analysis, appropriateness of corrective actions, and need for higher level involvement. When this review is conducted at an area or district office, a report describing ES office performance within the area or district jurisdiction must be communicated to the SWA on a quarterly basis.(2) Numerical appraisal at the SWA level must be conducted as follows:(i) Performance must be measured on a quarterly basis against planned service levels as stated in the State Plan. The State Plan must be consistent with numerical goals contained in ES office plans.(ii) To appraise these key numerical activities/indicators, actual results as shown on ETA Form 9172, or any successor report required by the Department must be compared to planned levels. Differences between achievement and plan levels must be identified.(iii) The SWA must review statewide data and performance against planned service levels as stated in the State Plan on at least a quarterly basis to identify significant statewide deficiencies and to determine the need for additional analysis, including identification of trends, comparisons to past performance, and attainment of State Plan goals.(iv) Results of numerical reviews must be documented and significant deficiencies identified. A corrective action plan as described in paragraph (a)(5) of this section must be developed to address these deficiencies. These plans must be submitted to the ETA Regional Office as part of the periodic performance process described at § 658.603(d)(2) .(3) Non-numerical (qualitative) appraisal of ES office activities must be conducted at least annually as follows:(i) Each ES office must assess the quality of its services to applicants, employers, and the community and its compliance with Federal regulations.(ii) At a minimum, non-numerical review must include an assessment of the following factors:(A) Appropriateness of services provided to participants and employers;(B) Timely delivery of services to participants and employers;(C) Staff responsiveness to individual participants and employer needs;(D) Thoroughness and accuracy of documents prepared in the course of service delivery; and(E) Effectiveness of ES interface with external organizations, such as other ETA-funded programs, community groups, etc.(iii) Non-numerical review methods must include:(A) Observation of processes;(B) Review of documents used in service provisions; and(C) Solicitation of input from applicants, employers, and the community.(iv) The result of non-numerical reviews must be documented and deficiencies identified. A corrective action plan addressing these deficiencies as described in paragraph (a)(6) of this section must be developed.(v) The result of ES office non-numerical appraisal, including corrective actions, must be communicated in writing to the next higher level of authority for review. This review must cover thoroughness and adequacy of ES office appraisal, appropriateness of corrective actions, and need for higher level involvement. When this review is conducted at an area or district level, a report summarizing local ES office performance within that jurisdiction must be communicated to the SWA on an annual basis.(4) As part of its oversight responsibilities, the SWA must conduct onsite reviews in those ES offices which show continuing internal problems or deficiencies in performance as indicated by such sources as data analysis, non-numerical appraisal, or other sources of information.(5) Non-numerical (qualitative) review of SWA ES activities must be conducted as follows:(i) SWA operations must be assessed annually to determine compliance with Federal regulations.(ii) Results of non-numerical reviews must be documented and deficiencies identified. A corrective action plan addressing these deficiencies must be developed.(6) Corrective action plans developed to address deficiencies uncovered at any administrative level within the State as a result of the self-appraisal process must include:(i) Specific descriptions of the type of action to be taken, the time frame involved, and the assignment of responsibility.(ii) Provision for the delivery of technical assistance as needed.(iii) A plan to conduct follow-up on a timely basis to determine if action taken to correct the deficiencies has been effective.(7)(i) The provisions of the ES regulations which require numerical and non-numerical assessment of service to special applicant groups (e.g., services to veterans at 20 CFR part 1001 -Services for Veterans and services to MSFWs at this part and part 653 of this chapter), are supplementary to the provisions of this section.(ii) Each State Administrator and ES office manager must ensure their staff know and carry out ES regulations, including regulations on performance standards and program emphases, and any corrective action plans imposed by the SWA or by the Department.(iii) Each State Administrator must ensure the SWA complies with its approved State Plan.(iv) Each State Administrator must ensure to the maximum extent feasible the accuracy of data entered by the SWA into Department-required management information systems. Each SWA must establish and maintain a data validation system pursuant to Department instructions. The system must review every local ES office at least once every 4 years. The system must include the validation of time distribution reports and the review of data gathering procedures.81 FR 56352, Aug. 19, 2016, as amended at 85 FR 629, Jan. 6, 2020 81 FR 56352, 10/18/2016; 85 FR 629, 2/5/2020