Patent Quality Metrics for Fiscal Year 2017 and Request for Comments on Improving Patent Quality Measurement

AGENCY:

United States Patent and Trademark Office, Commerce.

ACTION:

Request for comments.

SUMMARY:

The United States Patent and Trademark Office (USPTO) is revising its patent quality metrics to better identify quality-related issues and more clearly communicate its quality measurements to the public. The new patent quality metrics are part of the USPTO's Enhanced Patent Quality Initiative (EPQI), which was launched in 2015 to engage patent stakeholders in enhancing patent quality. As part of the Enhanced Patent Quality Initiative, the prior patent quality metrics have been reassessed, and new patent quality metrics are now being designed for adoption for fiscal year 2017. The new patent quality metrics for use in fiscal year 2017 are planned to focus on the correctness and clarity of Office actions and will be applied through a newly unified review process using a standardized review form that will permit data from a significantly larger number of finished product quality reviews conducted at the agency to be aggregated and mined for information. The USPTO will also mine data on transactions during patent prosecution (e.g., the types of actions taken by the applicant and the USPTO) to assess examination processes and identify potential quality issues requiring further study. The review process will apply the new quality metrics and standardized form to increase the accuracy, consistency, transparency, clarity, and simplicity of USPTO quality review procedures. The USPTO is seeking comment from its stakeholders on further improvements to the changes proposed herein.

DATES:

Comment Deadline Date: To be ensured of consideration in the development of the next iteration of metrics, written comments must be received on or before May 24, 2016.

ADDRESSES:

Comments should be sent by electronic mail message over the Internet addressed to: QualityMetrics2017@uspto.gov. Comments may also be submitted by postal mail addressed to: Mail Stop Comments—Patents, Commissioner for Patents, P.O. Box 1450, Alexandria, VA, 22313-1450, marked to the attention of Michael Cygan, Senior Legal Advisor, Office of Patent Legal Administration, Office of the Deputy Commissioner for Patent Examination Policy.

Although comments may be submitted by postal mail, the Office prefers to receive comments by electronic mail message over the Internet because sharing comments with the public is more easily accomplished. Electronic comments are preferred to be submitted in plain text, but also may be submitted in ADOBE® portable document format or MICROSOFT WORD® format. Comments not submitted electronically should be submitted on paper in a format that facilitates convenient digital scanning into ADOBE® portable document format.

Timely filed comments will be available for public inspection at the Office of the Commissioner for Patents, currently located in Madison East, Tenth Floor, 600 Dulany Street, Alexandria, Virginia 22314. Comments also will be available for viewing via the Office's Internet Web site ( http://www.uspto.gov/patent/laws-and-regulations/comments-public/comments-improving-patent-quality-measurement ). Because comments will be made available for public inspection, information that the submitter does not desire to make public, such as an address or phone number, should not be included in the comments. It would be helpful to the USPTO if comments included information about: (1) The name and affiliation of the individual responding; and (2) an indication of whether the comments represent views of the respondent's organization or are the respondent's personal views.

FOR FURTHER INFORMATION CONTACT:

Michael T. Cygan, Senior Legal Advisor, at (571) 272-7700. Inquiries regarding this notice may be directed to the Office of Patent Legal Administration, by telephone at (571) 272-7701, or by electronic mail at PatentPractice@uspto.gov.

SUPPLEMENTARY INFORMATION:

I. Background

Prior to fiscal year 2005, the USPTO quality metric was solely directed to the correctness of the final output of the examination process that would result in a patent: An allowed application. During fiscal years 2005 through 2009, the USPTO expanded its review efforts, employing two official metrics of examination quality: (1) The correctness of the examiner's determination of allowance of an application; and (2) the quality of the actions taken during the course of examination. In fiscal year 2010, the first metric was modified to include final Office actions, and the second metric was modified to focus on the quality of non-final Office actions during prosecution. All quality analysis was performed by random selection of actions for review by a dedicated Office of Patent Quality Assurance (OPQA) team of reviewers, which reviewed each selected action to determine whether all required claim rejections were properly made in compliance with the patent statutes.

In 2011, based on stakeholder input, the USPTO adopted a new “Composite Quality Metric” for fiscal years 2011-2015 to track performance of those aspects that affect quality and provide a single comprehensive metric representing the overall state of patent examination quality. The Composite Quality Metric was composed of seven total factors: (1) The final disposition review, (2) the in-process review, (3) the first action on the merits (FAOM) search review, (4) the complete FAOM review, (5) the external quality survey, (6) the internal quality survey, and (7) an aggregation of five factors from the USPTO's Quality Index Report (QIR). The first four factors continued the USPTO's focus on the statutory compliance of work product; i.e., the correctness of the Office actions. The first four factors were derived from the results of reviews of randomly selected Office actions that were conducted by OPQA. These reviews continued the USPTO's focus on the statutory compliance of work product; i.e., the correctness of the Office actions, with only a basic assessment of whether the examiner had sufficiently set forth his or her position for any claim rejections. The next two factors were derived from surveys that assessed both internal and external stakeholder views on USPTO quality. The final factor was based on the USPTO's QIR, which measures the degree to which actions in the prosecution of patent applications reveal trends indicative of quality concerns and uses a statistical analysis of occurrences of certain types of events (e.g., reopening after final Office actions, consecutive non-final Office actions, consecutive restriction requirements) based on data available through the USPTO's Patent Application Locating and Monitoring (PALM) system. Performance in the overall Composite Quality Metric and in each of the component metric factors has been published on the USPTO dashboard Web site on a quarterly basis. The information from the Composite Quality Metric has been used to identify trends and areas of concern and to target those areas in need of increased training and/or resources.

On February 5, 2015, the USPTO launched the Enhanced Patent Quality Initiative to improve the quality of patents issued by the USPTO. This initiative began with a request for public comments on a set of six proposals outlined in a Federal Register Notice. See Request for Comments on Enhancing Patent Quality, 80 FR 6475 (Feb. 5, 2015). The USPTO also held a two-day “Quality Summit” on March 25 and 26, 2015, at the USPTO headquarters in Alexandria, Virginia, to discuss the quality concerns of patent stakeholders and to receive feedback on the USPTO's proposals. Following the Quality Summit, the USPTO has continued its engagement with the public through numerous roadshows, events, and stakeholder meetings to further refine the steps that may be taken to improve quality.

The Enhanced Patent Quality Initiative targets three pillars of patent quality: (1) Excellence in work products; (2) excellence in measuring patent quality; and (3) excellence in customer service. In furtherance of the second pillar of patent quality, the USPTO is focusing on improving the internal metrics used to evaluate patent examination quality and on improving the communication of its patent examination quality measurements to the public. Through this initiative, the USPTO has received numerous comments on establishing appropriate quality metrics. The USPTO has considered all of the comments received through the Summit, the Federal Register Notice, and numerous quality outreach events. Based on the information received to date, the USPTO has identified key aspects of quality measurement essential to developing more effective quality metrics.

First, the clarity of the examiner's determinations and the rationale underlying the decisions made in Office actions is an important part of overall patent examination quality and should be emphasized in reviews of USPTO work product. Second, individual metrics that clearly reflect individual aspects of USPTO work product would better communicate patent quality than a single quality composite number that combines scores from unrelated sources such as surveys, procedural efficiency statistics, and substantive patentability compliance reviews. Third, improving the granularity of work product quality measurement to monitor compliance with each statutory provision and enable meaningful data at the work group and art unit level is highly desirable for providing targeted training resulting in greater consistency. Fourth, monitoring the process of examination, i.e., the type and number of actions taken during prosecution as reflected in the QIR, remains a high priority that is best used to spot unusual trends or occurrences that deserve further attention. Lastly, capturing a larger number of finished product quality reviews conducted at the agency and using a standardized review form will lead to a significantly greater number of data points, which will allow for greater consistency in the review of application quality within the Patents Organization. More information on the public comments received on the metrics, and how those are being used to identify improvements to the metrics, is available at http://www.uspto.gov/patent/initiatives/quality-metrics. In view of these guideposts, a new set of metrics is now being proposed to incorporate these and other improvements to the collection of data and reporting of metrics.

II. Improving Measurement of Patent Examination Quality

As the next step in advancing the second pillar of the Enhanced Patent Quality Initiative, the proposed fiscal year 2017 patent quality metrics refocus the USPTO's measurement of the quality of the work products produced from first Office action through final disposition. The proposed metrics continue to assess the correctness of an examiner's determinations in a given Office action with increased attention on assessing whether the examiner clearly set forth his or her reasoning in a given Office action. In addition, the Office will continue to review the transactions taken during patent prosecution through the QIR, but this information will be used to identify the need for further investigation rather than being measured against a goal. Additionally, the USPTO is changing its reporting of the quality metrics to provide simpler and clearer communication of results to the public.

A. Measurement of Statutory Compliance and Clarity in Work Products

The patent quality metrics of work product proposed here for fiscal year 2017 provide a tighter focus on measuring two foundational characteristics of patent examination: Statutory compliance and clarity of decision making in Office actions. These proposed patent quality metrics continue to measure correctness of actions in terms of their compliance with each of the statutory requirements for issuance of a patent. To this end, a sampling of Office actions will continue to be reviewed both for improperly made rejections and for failure to make rejections where required by statute. The substantive review items will also include other items, for example, the propriety of the examiner's search, any interpretation of claim language under 35 U.S.C 112(f), any determination that an action is made final, any restriction or election of species requirement.

Furthermore, the new metrics greatly enhance the review of the clarity of the components of Office actions by including new clarity review items specifically designed for each of the substantive patentability determinations made in Office actions. For example, when reviewing an Office action containing an obviousness rejection under 35 U.S.C. 103, the review items consider not only whether the obviousness rejection was proper, but also whether the statement of the rejection mapped the elements identified in the prior art to the claim limitations, and whether the statement of the rejection explained the reasons for the rejection in a clear manner. The new clarity review items will also include, for example, items directed to the sufficiency of the recordation of any interview and the propriety of any reasons for allowance of an application.

For fiscal year 2017, the USPTO is proposing to capture the correctness and clarity review items with a single standardized review form as a repository for all of the review items, replacing the review-specific forms used in the 2011-2015 Composite Quality Metric. The review questions on such a standardized form, colloquially referred to as the “Master Review Form,” is planned to be used by all USPTO reviewers for finished product quality reviews of actions at every stage of prosecution. This Master Review Form will contain the above-described criteria for recording correctness for each of the substantive patentability requirements and for recording the clarity of each of those decisions and the supporting rationales set forth in the Office action under review. The full list of correctness and clarity items in the draft proposed version of the Master Review Form is available for viewing at http://www.uspto.gov/patentquality. The USPTO welcomes and appreciates feedback on the elements of this form through this notice, and will use the input to help finalize the Master Review Form that will be deployed throughout the USPTO in fiscal year 2017.

This draft proposed “Master Review Form” was developed as part of the Clarity and Correctness Data Capture program, which is part of the USPTO's Enhanced Patent Quality Initiative. The Clarity and Correctness Data Capture Program has been instituted to better capture the data produced through the different types of reviews within the Patents Organization. Historically, reviews have been performed not only by the quality assurance team, but also by other Technology Center personnel, with each reviewing area setting its own reviewing criteria. Moreover, the only work product reviews recorded for identification of trends were those undertaken by the Office of Patent Quality Assurance. The Master Review Form is designed to provide standardized reviewing criteria for quality reviews of finished work product. Through application of standardized reviewing criteria, the USPTO can better leverage the results from the many levels of review conducted at the agency. The improvements to the data capture process will enable meaningful data analysis at a more granular level than previously possible, permitting valid inferences to be drawn at the workgroup and art unit levels. Through this process, the USPTO and the stakeholders in the patent system will be able to gain a greater understanding of the state of patent prosecution and to work better together towards its improvement.

B. Measurement of Transactions During Patent Prosecution

A further aspect of the new patent quality metrics will be the leveraging of the data representing the thousands of transactions made by the USPTO during prosecution to reveal information on the quality of the patent prosecution process itself. Transactions during prosecution, such as restrictions, first Office actions, and allowances, are monitored through the USPTO's PALM system. The USPTO monitors many of these transactions through its QIR. Since 2011, the USPTO has included some of these transactions, such as the number of occurrences of consecutive non-final rejections, as part of its reported quality data. For the proposed 2017 quality metrics, transactional data from the QIR will be used to identify information that can be used to prevent reopening of prosecution, reduce rework, and improve the consistency of decision making throughout the USPTO. Key indicators of the efficiency of prosecution will be instances of reopening of prosecution and repeated non-final Office actions, as well as other instances of rework (e.g., consecutive final Office actions, consecutive restrictions). These indicators do not, by themselves, provide a numerical measure of quality. Rather, these indicators will reveal trends and outlier behavior that will draw attention to potential quality concerns.

C. Clearer Reporting of the Metrics

In presenting the results of the quality data, the USPTO will seek to further improve the usefulness and transparency of our quality reporting and to communicate the results in a clear and simple manner. The 2011-2015 Composite Quality Metric, which combined seven different quality variables into a single composite number, will be discontinued. The Quality Index Report will be used to identify potential areas of concern, rather than as providing a single, reportable number. While internal and external surveys will still be performed, the results will not be part of the quality metric, but instead will serve as independent checks on the quality metrics.

D. Refinement of Proposed Quality Metrics in FY 2016

Fiscal year 2016 will represent a transitional period for the quality metrics, emphasizing the fine-tuning of the fiscal year 2017 patent quality metrics. The USPTO will test and refine its proposed Master Review Form. This Master Review Form will contain new items, such as additional clarity review items, that will require a period of data collection to create numerical baselines for these items. The Master Review Form will initially be used in targeted reviews to determine the effectiveness of each individual clarity and correctness review item. The transactional data from the QIR will also be reviewed during 2016 to optimize the data analysis therein. Stakeholder comments on the Master Review Form in response to this notice will also form an important part of the process of optimizing the components of the patent quality metrics. During this transitional period, the information gleaned during fiscal year 2016 will be used to produce a finalized set of quality metrics for fiscal year 2017 that will represent the next phase of quality measurement, analysis, tracking, and reporting at the USPTO.

III. Feedback Sought on Improving Metrics of Patent Examination Quality

The USPTO seeks input and comments from the public through this notice and through public outreach on the following:

(1) Is the USPTO moving in the right direction by choosing to focus on two core metrics: A work product metric representing correctness of actions, and a clarity metric that more thoroughly explores the sufficiency of the examiner's reasoning in an Office action, thus moving away from the prior goal-based quality “score” that reflected not only quality of work product but also results of surveys, used to discover both internal and external stakeholder opinions, and QIR process indicators? Which of the proposed clarity and correctness review items in the proposed standardized “Master Review Form,” available at http://www.uspto.gov/patentquality,, should be used as the key drivers of patent examination quality metrics?

(2) How can patent metrics best provide objective, rather than subjective, measurements of quality-related features in clarity and correctness reviews?

In addition to the three questions posed above, the USPTO welcomes comments on any and all areas of quality measurement. Suggestions for rephrased or additional quality metrics review items, especially clarity indicators, are welcomed. The USPTO will consider all submitted comments as it develops the next iteration of quality metrics.

For the most current information on this and other patent quality initiatives, please visit the Enhanced Patent Quality Initiative micro site at http://www.uspto.gov/patentquality.