Agency Information Collection Activities: Proposed Collection; Comments Request-Third Access Participation Eligibility and Certification Study Series (APEC III)

AGENCY:

Food and Nutrition Service (FNS), USDA.

ACTION:

Notice.

SUMMARY:

In accordance with the Paperwork Reduction Act of 1995, this notice invites the general public and other public agencies to comment on this proposed information collection. This is a reinstatement, with change, of a previously approved collection for which approval has expired (OMB Number 0584-0530, Expiration Date: 08/31/2015); for the Third Access Participation Eligibility and Certification Study Series (APEC III).

DATES:

Written comments must be received on or before October 11, 2016.

ADDRESSES:

Comments are invited on: (a) Whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility; (b) the accuracy of the agency's estimate of the burden of the proposed collection of information, including the validity of the methodology and assumptions that were used; (c) ways to enhance the quality, utility, and clarity of the information to be collected; and (d) ways to minimize the burden of the collection of information on those who are to respond, including use of appropriate automated, electronic, mechanical, or other technological collection techniques or other forms of information technology.

Comments may be sent to: Devin Wallace-Williams, Ph.D., Food and Nutrition Service, U.S. Department of Agriculture, 3101 Park Center Drive, Alexandria, VA 22302. Comments may also be submitted via fax to the attention of Devin Wallace-Williams, Ph.D. at 703-305-2576 or via email to Devin.Wallace-Williams@fns.usda.gov. Comments will also be accepted through the Federal eRulemaking Portal. Go to http://www.regulations.gov,, and follow the online instructions for submitting comments electronically.

All written comments will be open for public inspection at the office of the Food and Nutrition Service during regular business hours (8:30 a.m. to 5:00 p.m. Eastern Standard Time Monday through Friday) at 3101 Park Center Drive, Alexandria, Virginia 22302.

All responses to this notice will be summarized and included in the request for Office of Management and Budget approval. All comments will be a matter of public record.

FOR FURTHER INFORMATION CONTACT:

Requests for additional information or copies of this information collection should be directed to Devin Wallace-Williams, Ph.D. at 703-457-6791.

SUPPLEMENTARY INFORMATION:

Title: Third Access, Participation, Eligibility, and Certification Study Series (APEC III).

Form Number: Not applicable.

OMB Number: 0584-0530.

Expiration Date: Not Yet Determined.

Type of Request: Reinstatement, with change, of a previously approved collection for which approval has expired.

Abstract: The purpose of this third study on Access, Participation, Eligibility, and Certification (APEC III) is to provide the Food and Nutrition Service (FNS) with key information on the annual error rates and erroneous payments for the National School Lunch Program (NSLP) and School Breakfast Program (SBP) in school year (SY) 2017-2018. In addition, APEC III will identify School Food Authority (SFA), school, and student/household characteristics that may be related to error rates, and identify strategies and actionable guidance for reducing errors. Specifically, the four study objectives are:

• Objective 1: Generate a national estimate of the annual amount of erroneous payments based on School Year 2017-2018 by replicating the APEC methodology.
• Objective 2: Provide a robust examination of the relationship of student (household), school, and SFA characteristics to error rates.
• Objective 3: Conduct a sub-study on the differences in error rates among SFAs using different implementation strategies in their school meals programs.
• Objective 4: Perform qualitative analyses examining the reasons for erroneous payments.

Consistent with APEC methodology, APEC III will collect data to address the study objectives using a multistage-clustered sample design, which will include:

• A nationally representative sample of SFAs in the contiguous 48 states and the District of Columbia;
• A stratified sample of schools within each SFA (i.e. sampling from SFAs with Community Eligibility Provision (CEP) schools and from SFAs without CEP schools independently to ensure proportional representation in the final sample); and
• A random sample of students (households) within each sampled school that applied for free and reduced-price meals (including denied applicants), were categorically eligible for free meals, or were directly certified for free meals.

APEC III will collect data via in-person visits to SFAs, schools, and households to measure certification, aggregation, and meal claiming errors. Data collection will include (a) abstraction from income eligibility applications, categorical eligibility records and CEP records for determining the identified student percentage (ISP); (b) abstraction of meal count and claiming records from SFAs, schools and FNS administrative data; (c) an SFA director survey; (d) school meal observations; and (e) household surveys. Abstraction of income eligibility data and household surveys will take place three times during the study year to ensure coverage of applicants from different times during the year. APEC III data collection will also include qualitative data collection to help better understand the factors that contribute to errors, including an SFA director interview, a cafeteria manager interview and in-depth interviews with select households. Finally, administrative meal participation data (data on the number of meals served and claimed for sampled students) will be collected as well.

To measure certification error in non-CEP schools due to administrative errors, APEC III will independently determine certification status based on abstracted application data to assess errors in the SFA determination of certification status. To measure certification error in non-CEP schools due to household reporting errors, APEC III will independently determine certification status based on household survey data. This independent determination will be compared to certification status based on data reported on the application. To measure meal claiming errors, APEC III will conduct observations of a sample of meals served to students to confirm that meals claimed for reimbursement meet the meal pattern requirements. To measure aggregation error APEC III will abstract meal count and claiming records from different sources (school, SFA, State) for a target month, and identify discrepancies in data reported at each stage of the meal counting and claiming process. The following describes the types of error:

1. Certification errors

• Certification errors occur when students are certified for levels of benefits for which they are not eligible. Specifically, the student is certified for the wrong meal eligibility category.
• Because each meal eligibility category is reimbursed at different rates, an error in certification results in an incorrect level of benefit being paid to the SFA—either an overpayment or underpayment.
• Certification error may result from administrative error on the part of the SFA during application review or it can result from a household reporting error.
• Certification errors contribute the largest share to the total erroneous payments.

2. Meal claiming errors

• Meal claiming errors occur when there is an improper classification of meal reimbursement status based on meal components served.
• In schools operating with offer versus serve, including all high schools, the student may select fewer meal components/food items and still have a reimbursable meal (provided all components are offered to the student).
• In schools that are not operating under “offer versus serve,” a complete meal must contain all meal components required under the breakfast or lunch meal patterns.

3. Aggregation errors

• Aggregation errors occur in the process of counting, consolidating, and claiming the number of meals served in a given month (by claiming category—free, reduced priced, or paid)
• This occurs in the transmission of meal count and claim data between school, SFA, State and USDA for reimbursement.

The sample will include schools participating in the CEP and non-CEP schools. In summary, CEP allows school districts, individual schools, or groups of schools to offer breakfasts and lunches at no charge to all students if at least 40 percent of their students are “Identified Students”—that is, approved for free meals without an application based on participation in programs such as the Supplemental Nutrition Assistance Program (SNAP) or Temporary Assistance for Needy Families (TANF). In CEP, all meals are free to students. However, the percentage of meals claimed at the free and paid reimbursement level is determined by the Identified Student Percentage (ISP). Thus, the procedures for measuring certification errors in CEP will be focused on independently verifying the ISP and the claiming percentages for free and paid meals.

The analysis plan includes four components: Calculating error and erroneous payment estimates, comparisons to APEC I and APEC II estimates, quantitative and qualitative analyses to identify factors associated with errors, and developing an error forecasting model. The calculation of estimates from APEC III will include the incidence of error, the total dollar amount of error and the dollar based error rate. The comparisons to prior APEC studies will include tests for significant changes over time. The quantitative and qualitative analyses will examine the sources and causes of errors with a focus on identifying potential policy options for reducing errors. Finally, the estimation modeling will provide both State and National models for estimating errors using econometric forecasting and Bayesian approaches, and small area estimation models (for State level estimates).

Affected Public: Individuals/Households, State, Local, or Tribal Government, and Businesses and Other for Profit and Not for Profit Organizations. Respondent groups identified include: (1) Child Nutrition (CN) State agencies, (2) School Food Authorities (SFAs), (3) Schools (both CEP schools and non-CEP schools), and (4) parents/guardians of sampled students that are either certified to receive a free or reduced price meal or who applied for but were denied benefits in School Year (SY) 2017-18.

Estimated Number of Respondents: The total estimated number of respondents is 9,456. This includes 7,606 responding program participants, (b) 1,824 non-responding program participants, and (c) 26 program non-participants. The responding program participants include: 44 State CN agency administrators; 44 State CN data managers; 275 directors at SFAs; 275 staff at SFAs; 275 data managers at SFAs; 625 school principals; 625 school staff; 625 school cafeteria managers; and 4,818 parents or guardians of sampled students. The number of SFA Directors, Cafeteria Managers and parents or guardians that will also complete the qualitative in-depth interviews are included in the counts. Non-responding program participants include: 62 directors at SFAs, 156 school principals, and 1,606 parents or guardians of sampled students. Program non-participants, as part of cognitive pretesting, include: 9 SFA Director Survey Pre-test participants; 3 SFA Director In-Depth Interview Pre-test participants; 2 Cafeteria Manager In-Depth Interview Pre-test participants; 9 Household Survey Pre-test participants; and 3 Household In-Depth Interview Pre-test participants.

Estimated Frequency of Responses per Respondent: The estimated frequency of response across the entire collection is 6.25. For the respondents, the estimated frequency of response is estimated at 7.01 annually, while the frequency for non-respondents is estimated at 3.09 annually. Administrators at State CN agencies will be contacted up to two times: (1) Initial study contact and (2) a one-time data request for meal count and claiming data submitted by the sampled SFAs for the SY 2017-2018. Data managers at the State CN agencies will be expected to provide a response to the one-time data request.

The SFAs (SFA directors, staff at the SFAs, and/or data managers at the SFA) will be contacted up to eight times for: (1) Study notification and request for the verification of administrative data (2) to complete a telephone pre-visit interview; (3) an on-site visit to abstract records; (4) a telephone contact to ask for any additional income eligibility applications for new students enrolled during phase 2 of data collection; (5) a telephone contact to ask for any additional income eligibility applications for new students enrolled during phase 3 of data collection; (6) a request for administrative data submitted to the State CN agency; (7) a request to complete a web-based SFA Director Survey; and (8) a telephone contact with a sub-set of 60 SFA Directors that complete the SFA Director Survey to complete a qualitative in-depth interview.

Schools (principals, staff, and/or cafeteria managers) will be contacted up to three times for: (1) Study notification; (2) to complete a pre-visit telephone interview to help prepare for the in-person data collection visit; and (3) an onsite data collection visit to abstract meal count and claiming records, conduct observations of meal service and to complete a brief interview with the cafeteria manager.

Parents or guardians of sampled households will be contacted up to three occasions for: (1) Recruitment; (2) to complete a one time in-person household survey; and (3) to complete an in-depth phone interview (with a subset of 60 households that completed the Household Survey).

There will be approximately 62 non-responding SFAs, 156 non-responding schools, and 1,606 non-responding households. The burden for non-respondents is outlined in the table that follows, and includes the time to review introductory materials and respond to the follow up contact call, as well as data collection activities.

Program non-participants are contacted only once for the pretesting of survey instruments.

Estimated Total Annual Responses: The total estimated number of responses for data collection is 59,133. This includes 53,505 for respondents and 5,628 for non-respondents.

Estimated Time per Response: The estimated time per response is 14.76 minutes (0.246 hours) for respondents, and 2.94 minutes (0.049 hours) for non-respondents. The estimated time of response across the entire collection is 13.62 minutes (0.227 hours). The estimated time of response varies from 1 minute to four hours depending on respondent group, as shown in the table below.

Estimated Total Annual Burden on Respondents: The total public reporting burden for this collection of information is estimated at 13,445 hours (annually). The estimated burden for each type of respondent is given in the table below.