Improving Program Design

Executive Summary

Federal programs are often a story of good intentions gone awry.
Consider, for instance, the case of Supplemental Security Income (SSI)
recipients who are also eligible for food stamps. Congress, hoping to
improve service to, and participation of, those SSI beneficiaries,
concluded that the Departments of Agriculture and Health and Human
Services should coordinate the delivery of food stamps at Social
Security Administration (SSA) offices. Subsequently, however, the
General Accounting Office (GAO) found that this design resulted in
duplicated efforts and poor service.

Another flawed program design is the federal student loan program which
has guaranteed $142 billion in loans, paid $35 billion in interest, and
lost $19 billion in defaulted loans since 1965, yet it generates
hundreds of millions of dollars in profit every year for private
corporations. Poor program design also contributed to lax federal
regulation of the savings and loan industry, which will cost taxpayers
some $300 billion before the crisis ends.  More recently, the Federal
Aviation Administration struggled to keep up with rapid growth in
airline service because of serious institutional constraints, prompting
some experts to recommend transforming it into a government corporation.

These experiences are not isolated. Many federal programs are badly
designed, ill-conceived and fatally flawed. Innumerable bureaucratic
horror stories publicized by journalists, pundits, politicians, and
civil servants have unfairly turned phrases like "good government" into
oxymorons.  [Endnote 1] GAO has identified 17 federal programs as
high-risk activities especially vulnerable to waste, fraud, abuse, and
mismanagement.[Endnote 2] The Office of Management and Budget (OMB)
currently has 104 programs on its high-risk list.[Endnote 3]

The number and severity of these problems has undermined the public's
trust in government. Perhaps not since the beginning of the Great
Depression have Americans shown such little confidence in Washington's
ability to provide the service that the nation deserves. And, unlike
Watergate, Iran-Contra, and other political scandals, the problem is not
rooted in the episodic vices and failures of individuals. Instead,
Americans are increasingly frustrated by the enduring deficiencies in
our government's structures and processes.

Often, government problems are rooted in the poor foundations upon which
public programs are built-- ambiguous goals, weak operational concepts,
and careless implementation design. Now, both program design and program
evaluation are enjoying heightened prominence with enactment of the
Government Performance and Results Act of 1993. It requires the
executive branch to produce both performance plans and performance
reports; these requirements should prompt agencies to foster better
program designs and more rigorous program evaluations.

Federal programs come in a wide assortment.  Sometimes, the federal
government provides cash payments to individuals, such as with Social
Security. At other times, it provides benefits only indirectly, such as
when it offers tax incentives for builders to construct low-income
housing. The particular method of delivery of public service (e.g.,
direct subsidies, tax incentives, loan guarantees, regulations, etc.)
should be tailored to the specific requirements and circumstances of the
constituency to be served.

To be sure, political considerations, both in the executive branch and
Congress, play a major role in influencing the shape and funding of most
federal programs. Nevertheless, these programs also should be rationally
formulated and designed. In fact, sound designs that are based on
objective criteria could yield programs that are more likely to win
administrative and legislative approval.

At present, the federal government lacks any systematic, disciplined
approach to program design.  In designing programs, officials all too
often fail to anticipate the unintended consequences. Program evaluation
in OMB and many agencies is less prominent than it once was. Yet we need
broadly applicable techniques for designing effective, cost-efficient
programs, especially in an era of fiscal austerity.  We also should
apply program design technology to existing programs as part of program
review, evaluation, and redesign processes. In time, the objective
criteria that we develop may help us better determine what services the
federal government should provide, and what we should discontinue. The
following are examples of such objective criteria:

 1. Identify who benefits and how they benefit.

 2. Define and evaluate alternate methods of program delivery.

 3. Examine program compatibility.

 4. Assess cost-effectiveness and efficiency.

 5. Evaluate consistency with accepted management principles.

 6. Ensure financial feasibility.

 7. Determine feasibility of implementation.

 8. Provide for program flexibility.

 9. Institute performance measurement and program evaluation.

 10.Build in cessation provisions.

Whatever criteria are chosen, the federal government needs to develop a
formal discipline of program design. The National Performance Review
recommends four presidential initiatives.

First, the President should direct the President's Management Council to
sponsor the development and publication of a comprehensive handbook that
would address the strengths and weaknesses of alternative program
designs. It should include a complete review of relevant literature, a
compilation of existing federal programs, a discussion of different
approaches, an assessment of comparative advantages, analyses of case
studies, and model designs for program types.

Second, the Council should designate one or two agencies to test program
design capability to determine its value and costs. For these pilot
projects, agencies might create modest program design offices--staffed
by individuals qualified in program design, management and
evaluation--to help senior officials design new programs and assess
existing ones.

Third, the Council should take steps to help Congress design better
programs. It should work with the Congressional Research Service (CRS)
so the CRS will include program design concepts in all of its relevant
training materials. In addition, the Council should encourage CRS, GAO,
the Congressional Budget Office, and key congressional staff to
participate in program design training.

Fourth, the Council should commission program design courses to educate
and train current and future policymakers, program designers, and
managers. Such education might include short seminars, a longer
residential course, the introduction of program design into existing
management development programs, and offerings at graduate schools of
public administration.


Endnotes

1. Ruby, Michael, "Rube Goldberg, R.I.P.,'' U.S. News & World Report
(August 16, 1994), p. 64.

2. See U.S. General Accounting Office, Reports GAO/HR-93-1 through
HR-93-17, Washington, D.C., December 1992.

3. Budget of the United States Government Fiscal Year 1994 (Washington,
D.C.: U.S. Government Printing Office, 1993), p. 107.

**************************************************

The Need for Change

In a speech at a conference in Atlanta some years ago, a federal
official told a story about a young attorney working on Capitol Hill who
was asked to draft a bill to regulate some aspect of the airline
industry. He started by looking up and copying the law that addressed
similar concerns with the railroads. Finding some provisions which did
not seem applicable to railroads, he did further research and discovered
that the railroad bill had been lifted 75 years earlier by another
attorney from a law passed early in the 19th century to regulate canals.
Justice Oliver Wendell Holmes said that "in the law, an ounce of history
is worth a pound of logic.''[Endnote 1] However, in the design of
federal programs, unfortunately both are often missing. Consider the
following 1981 exchange before a subcommittee of the House Committee on
Banking, Finance and Urban Affairs:

Chairman Blanchard: Have you made any conclusions as to where
governmental loans are an effective policy instrument and where they are
not? Where loan guarantees are an effective policy instrument and where
they are not? How they work, in what instances they work well and where
they don't, and where they can be effective and where they aren't?
Because we are looking at these in relation to other tools that the
government has to operate...Have there been any studies on...federal use
of direct spending versus loans versus loan guarantees versus the tax
code, and then the various types of tax preferences and their
effectiveness in different instances?

The witnesses responded that they were unaware of any such
literature.[Endnote 2] Earlier the Congressional Research Service
observed:

These program differences underline the ways in which the programs have
evolved. There have been no overall guidelines or framework within which
these programs are created. They have evolved over time to meet
particular needs and situations. This generally ad hoc framework has led
to a situation where, except for vast generalizations, it is hard to
make specific statements concerning the characteristics of federal loan
guarantees.[Endnote 3]

Is there a systematic, disciplined approach to program design in the
Federal Government? There is not. An extensive literature search and
consultation with experts inside and outside government reveal an
overwhelming consensus that the lack of attention to program design
means that federal activities are less effective and less economical
than they should be.  Program designs are frequently characterized by
insufficient rational analysis and failure to anticipate unintended
consequences. Often this can be attributed to the poor foundations upon
which the programs were built- ambiguous goals, weak operational
concepts and careless implementation design. It is rare to compare
programs across agency and department lines in terms of what they do and
how they do it. We do not have a methodology with which to compare them.

Some public policy/administration academics have attempted to develop a
rational, disciplined approach to program design. In 1985, A Workbook in
Program Design for Public Managers noted the absence of any preexisting
attempt to provide this discipline and recognized its own limitations,
stating ". . . at best, this workbook is but a beginning
effort.''[Endnote 4] In 1986, the classic text Politics, Position and
Power (Fourth Edition) proposed twelve criteria for evaluating the
design and organization of new or existing programs.[Endnote 5] In 1989,
the Urban Institute published a primer on program reviews that outlined
criteria for analyzing program alternatives.[Endnote 6]

While academic interest in program design seems to be growing, the
practical applications have been few.  "Over time, legislation has
suffered more and more from wishful thinking, sloppy drafting, and
woeful disregard for public-sector organizational principles.''[Endnote
7] A congressional effort to establish a program design discipline for a
segment of related governmental programs can be found in the 1988 public
law which elevated the Veterans Administration to departmental status.
The law provided for the establishment of a National Commission on
Executive Organization, generally patterned on the first Hoover
Commission. Its charter would have required ". . .  establish[ment of]
criteria for use by the President and Congress in evaluating proposals
for government corporations and government-sponsored enterprises and
subsequently overseeing their performance.''  [Endnote 8] The new
commission could have been activated by presidential initiative;
however, it was not.

Rigorous assessments of federal programs and their
designs were once an active responsibility of OMB;
however, current oversight responsibilities regarding
program design are cloudy. In OMB and many agencies,
program evaluation as a related discipline is also
less prominent now.

Both program design and program evaluation will become more prominent
under the new Government Performance and Results Act of 1993. The Act
requires performance plans and performance reports from the executive
branch; hence, agencies may be prompted to foster sound program designs
as well as rigorous program evaluation.

What are the consequences of the absence of a rational program design
process? Too often programs are designed without careful consideration
of the way that similar designs have worked previously or the potential
benefits of alternate designs.[Endnote 9] It is no wonder news media,
federal auditors, politicians and taxpayers are critical of a government
that too often creates federal programs that do not meet public
expectations. A consequence is sagging public confidence in the ability
of government to deliver.

For example, flawed program design accounts for poor public service in
food stamp application processing for Supplemental Security Income (SSI)
applicants and recipients. Congress believed that food stamp program
participation would increase, and more streamlined public service would
occur, if the Departments of Agriculture and Health and Human Services
coordinated the delivery of food stamp services at Social Security
Administration (SSA) offices. For SSI clients eligible and willing to
apply for food stamps at SSA field offices, SSA staff complete an
application and forward it to a state food stamp office for further
consideration.[Endnote 10] GAO concluded:

The process for taking food stamp applications at SSA has not worked
well...further, the process as designed results in duplication of effort
and poor service to clients...the food stamp applications used are
unnecessarily complex and lengthy.''  [Endnote 11]

There are many other examples of poor programs that have received public
criticism because they fail to meet various design criteria. These
include:

--Regulation of Savings and Loan Institutions.  The government intended
to use federal deposit insurance to subsidize the cost of loans and
increase the availability of mortgage money for home buyers. To avoid
the misuse of a government guarantee such as deposit insurance, the
government must establish an effective and independent supervisory
agency. The Federal Home Loan Bank Board lacked this independence from
the savings and loan industry that it was supposed to supervise. The
ultimate cost of that regulatory failure will amount to some $100
billion.[Endnote 12]

--Reduction of Crime. In attempting to reduce burglaries, the Law
Enforcement Assistance Administration tried to increase the
effectiveness of limited numbers of law enforcement officials through
large investments in sophisticated telecommunications in police
cruisers. These efforts were much less effective than Neighborhood Watch
programs which greatly increased the number of people participating in
crime surveillance at minimal cost.[Endnote 13] The consideration of
alternatives in the design of federal programs is all too rare.
Consequently the allocation of program resources often is not optimal in
accomplishing desired ends.

--Federal Catfish Research. The Senate has noted that catfish research
is conducted in at least four agencies: Fish and Wildlife Service
(Interior), Animal and Plant Inspection Service (Agriculture),
Agriculture Research Service, and Agriculture Cooperation Service.
Although a 1988 report by the Inspector General at the Department of
Commerce (itself conducting independent catfish studies) recommended
that the work be consolidated, it still has not been done.[Endnote 14]
Federal catfish research raises a variety of design questions. How
compatible are these research efforts with each other?  Are they
complementary or redundant? What are the aggregate costs for federal
catfish research and are they justified by the benefits?

--National Air Traffic Control. The Federal Aviation Administration
(FAA) is responsible for airline safety in the United States and
administers the nation's air traffic control system. Commentators have
repeatedly pointed to problems confronting the FAA as it works to keep
up with rapid growth in airline service and cope with federal budget
constraints and inflexibility in its institutional structure. Recently
the National Airline Commission recommended that the FAA become a
government corporation so that it can regain the capacity to carry out
its essential functions.[Endnote 15]

What is a program? In this context, a program is "a particular
combination of authority, organizations, resources and personnel
assembled to achieve specific public purposes.''[Endnote 16] At times,
the federal government's role is very direct, such as payment of cash to
Social Security beneficiaries. Alternatively the role can be indirect,
such as the establishment of tax incentives for the private sector to
provide low income housing.

In the field of public administration, two other levels of analysis have
been proposed. One analyzes a set of programs or a program cluster aimed
at a specific, common target group.[Endnote 17] For example, there are
many programs addressing the income maintenance needs of the poor.
Rather than considering a single program's design in isolation, the
preferred focus is to diagnose program effectiveness as a whole.

A second alternative is to focus on the generic tools of government
action, the techniques of social intervention used in particular public
programs.[Endnote 18] Reinventing Government identifies 36 such
mechanisms, ranging from grants to tax policies.[Endnote 19] In some
program designs, more than one tool may be used; therefore, it is
helpful to examine how well each tool works, either alone or in
combination with others.

Frequently, each of these perspectives merits consideration. For
example, consider a category of programs using more than one method of
delivery--job training. It can be delivered in many forms including tax
policy, grants, loans, loan guarantees, technical assistance,
information, referral, seed money, and jaw boning. Whatever combination
of delivery tools may apply to a specific public purpose, it would be
desirable to understand the programmatic impact of the tools
individually and collectively in order to design the best portfolio.

Why do we need an emphasis on program design?  Effective and efficient
delivery of programs is a major determinant of the public's confidence
in its government. Although the approval and funding of most federal
programs are inevitably shaped by political considerations (both in
Congress and the executive branch), the formulation and design of these
programs should be grounded in rational and valid public purposes. Sound
designs based on objective criteria can yield programs more likely to
win bureaucratic and political approval. A program design discipline
that can increase the likelihood of a program's success is especially
important in a time of especially austere fiscal constraints. In
addition, this program design discipline should be applied to existing
programs as part of program review and redesign processes.[Endnote 20]
The American public is willing to support programs if it is confident
the programs address issues important to them and the solutions have a
reasonable probability of success (e.g., health care reform).[Endnote
21]

************************************************

ENDNOTES

1. Knisely, Robert A., "Toward a Science of Program Design,'' speech at
ORSA-TIMS conference, Atlanta, Georgia, November 9, 1977.

2. Salamon, Lester M., Beyond Privatization: The Tools of Government
Action (Washington, D.C.: Urban Institute Press, 1989), p. 23.

3. U.S. Congress, House, Committee on Banking, Finance, and Urban
Affairs, Subcommittee on Economic Stabilization, Catalog of Federal Loan
Guarantee Programs, SN 052-070-04263-0 (September 1971).

4. Ruchelman, Leonard I., A Workbook in Program Design for Public
Managers (SUNY Press, 1985).

5. Seidman, Harold, and Robert Gilmour, Politics, Position & Power,
Fourth Edition (New York: Oxford University Press, 1986), pp. 338-340.

6. Hatry, Harry P., Kenneth P. Voytek, and Allen E.  Holmes, Building
Innovation into Program Reviews (Washington, D.C.: Urban Institute
Press, 1989).

7. Moe, Ronald C., "Let's Rediscover Government, Not Reinvent It,''
Government Executive, vol. 25, no. 6 (June 1993), p. 46.

8. Moe, Ronald C., Reorganizing the Executive Branch in the Twentieth
Century: Landmark Commissions, 93- 293 GOV (Washington, D.C.: Library of
Congress, Congressional Research Service, March 19, 1992), p. 50.

9. Hatry, et al., pp. 5-6.

10. The Food Stamp Act requires that food stamp applications be
forwarded immediately and in an efficient manner. Food and Nutrition
Service regulations define this to mean one working day.

11. U.S. General Accounting Office, Social Security:  Need for Better
Coordination of Food Stamp Services for Social Security Clients, HRD
92-92 (Washington, D.C.: U.S. General Accounting Office [GAO], September
1992), p. 5. See also NPR Accompanying Report, Department of Agriculture
(recommendation USDA07).

12. U.S. Congress, Congressional Budget Office, The Economic Effects of
the Savings & Loan Crisis (January 1992).

13. Knisely, Robert A., "Law School and the Design of Government,''
November 1985.

14. "Reinventing Catfish Research,'' Washington Post, Federal Page:
Short Takes (July 1, 1993), p. 21; and U.S. Department of Commerce,
Office of Inspector General, Office of Audits, Denver Regional Office,
NMF's Funding of the Stuttgart Catfish Farm is Not Warranted, Final
Audit Report No. D-197-8-022 (July 1988).

15. National Commission to Ensure a Strong, Competitive Airline
Industry, Outline of Issue:  Restructuring the Federal Aviation
Administration, Working Draft Report, pp. 8-9 (the draft reflects
commission deliberations of July 19, 1993). See also National Academy of
Public Administration, The Air Traffic Control System: Management by a
Government Corporation (Washington, D.C., March 1986); Jasper, Herbert
N., National Research Council, Transportation Research Board, "Appendix
B: Organizational Options for the Federal Aviation Administration,''
Winds of Change : Domestic Air Transport Since Deregulation; and
discussion in the NPR summary report and the NPR Accompanying Report
Department of Transportation (recommendation DOT08).

16. Salamon, Lester, p. 25.

17. Interview with Joe Wholey, Director, Public Administration,
University of Southern California, at Washington, D.C. campus, May 26,
1993.

18. Interview with Lester Salamon, Director, Public Policy, The Johns
Hopkins University, at National Performance Review, May 17, 1993.

19. Osborne, David, and Ted Gaebler, Reinventing Government: How the
Entrepreneurial Spirit is Transforming the Public Sector (Reading, MA:
Addison- Wesley Publishing Company, Inc., 1992).

20. See Hatry, et al.

21. "Seeking a Cure: Most Americans Pledge Sacrifice to Help Fix the
Health System,'' Wall Street Journal/NBC News Poll, Wall Street Journal
(March 12, 1993), p. A1.

***************************************************
Program Design Criteria

Whatever the purposes of any public program, its logic and internal
consistency can be exposed and analyzed using objective rational
criteria. The following is a set of criteria which could be applied
systematically when designing or redesigning programs.[Endnote 1]
Research and development to further the field certainly will refine and
augment this list.

1. Identify Who Benefits and How They Benefit

All program designs require clear, explicit identification of the
primary beneficiaries and the specific benefits they can expect. Without
this foundation, no rational program design methodology can be
sustained. Programs designed to solve particular problems should be
consistent with the missions of the agencies responsible. However, both
the problems addressed and the missions pursued should always be framed
in the context of who benefits.

Prioritize Beneficiaries. The public purposes and program goals must be
defined in terms of benefits to individuals and/or groups. There are two
levels of beneficiaries, primary and secondary, which must be reflected
in a program's design.

Primary beneficiaries benefit directly from a program. To illustrate,
the Department of Veterans Affairs serves veterans; the programs it
administers clearly define categories of veterans who qualify as
beneficiaries. Other programs' intended primary beneficiaries are not as
clear. Consider government programs allowing removal of trees from
federal lands. In this latter case the class of primary beneficiaries
may be the loggers--who would otherwise be unemployed--the logging
companies, the domestic and foreign timber consumers, or the
recreational users who object to despoiling the environment.  Secondary
beneficiaries are those individuals, groups of individuals, or
organizations benefiting indirectly from a program. For example, state
governments are the secondary beneficiaries of federal grants for
education.

Finally, a class of interested parties, not strictly beneficiaries of a
program, are stakeholders, such as taxpayers. The taxpayers deserve the
most effective and efficient design possible to maximize the return on
their investment in tax dollars.

Validate the Need. It is essential to verify that the need exists and
will persist. The program designers must be able to document that the
need is more than a short-term aberration. The determination of who
benefits and how they benefit should be validated by the intended
beneficiaries themselves, which is the essence of customer-driven
government. Federal-level involvement must also be justified. The
program must avoid addressing obsolete problems or needs that are more
effectively solved through state or local initiatives, through the
private sector, or by nonprofit organizations.

Determine Program Size and Scope. Program designers must also determine
the size and scope of the population to be served so the course of
action matches the problem. How many people are affected?  What are the
eligibility criteria? What are the characteristics of the affected
populations (geography, socio-economic factors, etc.)? What level of
diversity exists among the populations affected?  What is the minimum
program size necessary to achieve the results required?

If there is more than one problem to be addressed, the program designers
must identify each need and prioritize solutions to avoid unbalanced or
skewed designs. Social Security is an example of a program that has
well-defined beneficiaries and missions. It is designed to provide
income to individuals (and their eligible family members) who are
retired or disabled. The Social Security Administration has long viewed
its mission as the right check, to the right person, in the correct
amount, on time. Clear eligibility criteria enable reliable estimates of
client populations and size of benefits.

Inspire Public Confidence. Program design should give the program
credibility with the public. There is a common perception that the
political process does not always place program priorities on the
appropriate beneficiaries. Without strong public confidence, programs
are hindered from gaining approval and overcoming resistance during
implementation. A federal program should appear to make sense both in
intent and by design as a valid commitment of public resources.

2. Define and Evaluate Alternate Methods of Program Delivery

After clearly defining program goals and beneficiaries, the designers
should consider any alternate forms of program delivery. Not all public
needs must be satisfied by direct federal delivery of services.
Alternatives to direct government service include regulatory
requirements, government sponsored enterprises, tax incentives,
government guarantees, monitoring and enforcement, etc.

Different program delivery methods can often attain similar ends,
although costs and schedules are not likely to be the same. Education is
a prime example.  Education has been delivered in many forms such as
direct cash payments to veterans, categorical grants to states to fund
educational initiatives at the local level, block grants, tax
exemptions, privatization, etc.Two sources of potential alternatives are
the list of 36 governmental mechanisms that were identified by Osborne
and Gaebler in Reinventing Government, and the program tools identified
by Professor Lester Salamon of Johns Hopkins University.[Endnote 2]
Salamon suggests that the major tools of public action should be studied
in a well-structured framework:

Given the great proliferation of instruments that the public sector
uses, and the likelihood that this proliferation will continue as a
result of the privatization movement and the resource constraints under
which government is operating, the development of a body of knowledge
that can help guide the selection of tools and instruct program managers
about the consequences of these choices seems increasingly
worthwhile.[Endnote 3]

Review Against Assumptions. Alternatives must be reviewed explicitly
against fundamental assumptions.  What are the critical success factors
identified by the beneficiaries? How has success been measured and how
well does each program design address the needs identified? What is each
design's ability to serve people, to adapt to emerging technology, and
to use existing facilities? What are the possible unintended
consequences or adverse effects of each design mechanism? How do they
compare to the status quo?  Examples of programs which merit review
against their initial assumptions include job training, veterans health
care, and sundry entitlement programs.

3. Examine Program Compatibility

It is rare that any new program is so independent and autonomous that it
enters a domain totally unpopulated by preexisting efforts.
Consequently, program design should be mindful of compatibility and
complementarity with existing infrastructure or related programs.

Comparative Advantage. What does this program do that is unique? Would
another program or program design perform these functions better? What
would beneficiaries lose if this program did not exist?

Complementarity. Does the program complement other programs that serve
the same beneficiaries? Does the program needlessly duplicate or overlap
other programs at the federal, state, or local levels? Does the program
foster cooperation or does it create barriers between beneficiaries and
administrators? To illustrate, one-stop shopping, which provides
multiple services conveniently at one place, contributes to integrated
program delivery and is being considered by the Department of Labor for
workforce development.[Endnote 4]

Harmony. Is the program consistent with the mission of the agency which
would administer it? Does it create conflict in the agency? Is the
program more compatible with another agency's mission? Are another
agency's programs more interrelated and would that agency therefore be a
better fit? How easy is it to add the program to an existing
organizational structure? Will it replace one or more existing programs?
Does it create or reduce redundancies?

Catalyst. Does the program foster competition or does it encourage
monopolies? Does it serve as a catalyst?  Does the program design result
in the empowerment of people and communities so they have a vested
interest in how the program is run? Is there positive leverage with
other programs or destructive interference? Does the sum of all programs
encourage the right behavior or do perverse incentives motivate
dysfunctional behavior?

4. Assess Cost-Effectiveness and Efficiency

Program designers should apply cost-effectiveness and efficiency
criteria to the program as a whole and to specific program elements
(including organizational structures, program delivery, and
administrative support functions) to estimate:

--cost (total, marginal, and/or administrative, as a measure of
overhead) per unit of benefit (defined as program outputs or outcomes);
and

--productivity (unit of benefit per full time employee or field office).

Program designers also should consider the program's social efficiency;
i.e., how the program might be designed to achieve the greatest net
benefit to society as a whole. In this context, benefits and costs are
defined more inclusively than in analyzing cost-effectiveness, although
there probably will be some overlap. To determine efficiency, analysts
would employ the standard techniques of benefit-cost analysis to
estimate the net social benefits of one or more program designs. Steps
include:

--identifying major relevant impacts of the policy, including direct and
indirect impacts on the program's direct beneficiaries, other affected
groups, and society as a whole;

--categorizing costs and benefits for various affected groups; and

--quantifying dollar impacts.

In principle, the program design which is estimated to achieve the
highest net benefit should be selected. However, optimizing
distributional benefits (to either primary or secondary beneficiaries)
typically conflicts with optimizing aggregate social welfare. Some times
this may be appropriate (i.e., targeted programs). At other times some
might criticize the selection of sub-optimally efficient programs as
pork-barrel spending. Making explicit the trade-offs inherent in each
design can help inform the debate.

5. Evaluate Consistency with Accepted Management Principles

While the field of management is not a precise science, it does
represent an accumulation of knowledge useful in illuminating the
strengths and weaknesses of public programs. Using generally accepted
management principles to evaluate various program designs may help
identify both inherent deficiencies and alternative remedies. Where
violations of conventional wisdom are appropriate, the justifications
should be clear and convincing.

Among the numerous management principles applicable to program design to
be given particular emphasis are those consistent with the core values
of reinventing government, namely:

--serving customers,

--empowering employees,

--empowering communities to solve their own problems, and

--fostering excellence.

For example, program designers should consider the following reinvention
principles that improve the efficiency and effectiveness of their
programs:

--create a clear sense of mission,

--steer more, row less,

--delegate authority and responsibility,

--replace regulations with incentives,

--develop budgets based on outcomes,

--inject competition where appropriate,

--search for market, not administrative, solutions, and

--measure success by customer satisfaction.

6. Ensure Financial Feasibility

Given the increasingly strong drive to reduce the federal budget
deficit, and competing claims on limited funding, affordability is an
inevitable criterion for evaluating new and existing public programs.
Although not all programs have direct budgetary impact, many do.
Typically, cost calculations and estimates can be difficult to derive.
Accounting methods interact with funding streams and auditing
requirements in ways that are not always sensible or consistent.

Nevertheless, the program cost must be known if the government is to
avoid incurring unlimited liabilities. A program's budgetary costs can
be determined in a variety of ways, each of different interest to
different constituencies. Among the alternate cost elements that may be
relevant are:

--total life cycle costs: the full costs of creating, operating,
maintaining, and terminating the program from inception to conclusion
(where end points are reasonably defined);

--annual budget requirements: the amount that must be appropriated
annually to sustain/maintain the program throughout its lifetime;

--risk of cost overruns: the probability that actual program costs will
exceed approved budget levels including the expected cost of unbudgeted
contingent liabilities;

--revenue generation: the expected income return to the Treasury from
the program's operation; and

--debt management: the ratio of the funds collected or recaptured to the
amount owed the U.S.  Treasury.

7. Determine Feasibility of Implementation

Even the best designed program will not be effective in its execution if
fatal flaws of implementation are not considered during its design.
Unfortunately, those persons most familiar with implementation do not
usually play key roles in design, depriving policy officials of the
opportunity to obtain field validation and assessment before final
decisions on design. Without effective implementation, program
proponents will not see their goals successfully accomplished.Given the
number and types of federal programs, it is unrealistic to formulate
universal principles of implementation that apply exactly to all
programs. Yet it may be instructive to identify a few common pitfalls to
consider in the designs of many types of public service programs.

Foremost, implementation should be embedded in the program
design.[Endnote 5] Therefore, the more that programs explicitly consider
anticipated difficulties and conditions of implementation, the fewer the
surprises and the more likely program success.  Although detailed
operational plans are not usually developed during the program design
phase, at least some consideration in broad outline would be prudent
management and politics. However, implementation must not compromise or
obscure the program's basic purpose and integrity. If political
realities so dilute the program's integrity that the original intent is
no longer served, then managers should abort the design rather than
succumb to the temptation of just getting anything approved.

Second, it is imperative to identify clearly who is ultimately
responsible for the program to anticipate their capacity to administer
and to ensure accountability. We cannot know the barriers and
impediments to success without knowing clearly who (which
organization(s)) is charged with carrying out the program. An objective
assessment of the capabilities of the agencies through which the program
resources and responsibilities will flow could pinpoint potential
roadblocks, sources of conflict, and choke points that might hamper
implementation. And, as trite as it sounds, those responsible for
implementing a program must know what they are supposed to do. Policies,
if not clear, accurate, and consistent, may cause misunderstanding and
confusion, and confer discretion that may be roundly abused. For federal
programs to be implemented more effectively, we must ensure the
following:

(a) those bearing the major responsibility for implementation are aware
of what isactually happening in another unit of government, a field
agency, or the private sector;

(b) those responsible for implementation earnestly strive to achieve the
stated program goals and mission;

(c) policies and actions are decisively made and clearly transmitted;
and

(d) decisions and policies encourage creativity, flexibility, and
adaptability.

Third, an accurate description of the organizational network directly
affected by the program would identify the critical relationships and
points of coordination required or implied by program operations. This
might include a flow chart of transactions depicting key relationships.
Omissions and redundancies are more easily and economically remedied in
the planning stages than later.  Implementing policies and programs
requires a coordination of efforts between various groups and
organizations, particularly regarding complex policies or decisions.

Fourth, the timing of critical events is absolutely essential. Although
many alternate sequences of activities are possible, key deadlines may
not be discretionary (e.g., budget approvals, legislative requirements,
awarding of contracts, etc.) and certain schedules offer distinct
advantages over others. More programs suffer from overly ambitious
timetables than from unduly conservative ones; effective implementation
requires sufficient time.

Fifth, in designing programs, attention must be given to resources.
However, due to various political compromises and conflicts, sufficient
resources often are not available to implement a program properly.
Program designers may not be aware of inadequate resources or may choose
to ignore the problem. There are several important components of
resources critical to implementation, including:

(a) adequate staff with the appropriate knowledge (programmatic and
managerial);

(b) sufficient information on how a program or decision is to be
implemented and the support/approval of other necessary agencies and
participants;

(c) necessary authority to permit the programs or decisions to be
implemented; and

(d) facilities and equipment necessary to implement the program or
decision.

Sixth, prudent experimentation before committing the government to a
major public investment could help avoid expensive misadventures and
illuminate the pitfalls in uncharted domains. While pilot demonstrations
are not appropriate for all programs, they are highly desirable under a
wide range of conditions such as:

(a) high risk programs--where the consequences of ailure are
catastrophic or life threatening;

(b) where the economic investment is substantial enough to justify small
experiments;

(c) where there is significant complexity--either organizational or
programmatic (e.g., health care, homelessness, economic renewal);

(d) where there is great uncertainty surrounding the proper solutions;

(e) where there is intense public controversy or low political
consensus; and

(f) when there is no immediate urgency that precludes more deliberate
exploration of alternate approaches.

8. Provide for Program Flexibility

Perhaps the only universal element in all federal programs is the
certainty that even the most carefully formulated plans and policies
will not be realized exactly as designed. Given the inordinately long
gestation periods for many public programs, coupled with lengthy
implementation periods, it may be as long as five years before a program
may be mature enough to sustain a valid evaluation.

In the meantime, the conditions and constraints under which a program
operates may have changed significantly from when it was first
conceptualized.  Therefore, the design of programs should permit enough
management flexibility to allow agencies to adapt to external changes,
unforeseen circumstances, variations in resource levels, and schedule
changes.  In addition, program designs should free implementing
organizations to tailor programs to local circumstances. Where initial
rigidity in structure is necessary, waiver authority is essential to
accommodate exceptional cases and unanticipated conditions.[Endnote 6]

While no one can anticipate every possible contingency, the most likely
changes (e.g., budget erosion, schedule slippage, personnel shifts)
should not have catastrophic impacts on well-designed programs. Programs
dependent on critical, highly sophisticated, specialized, or scarce
resources (e.g., key individuals, unproven technology, rare expertise,
unique facilities) are fragile because they are vulnerable to
uncertainties in the availability, quality, and quantity of these
essential resources. For example, many attempts to modernize automated
information systems often under- or over-estimate the availability of
state of the art technology, resulting in either investments in obsolete
hardware/software or costly delays waiting for technology to catch up.

Two especially critical flexibilities involve schedule and budget. With
the enormous vagaries of when and what will survive the approval
processes both in the executive and legislative branches, and the
uncertainties of program execution, good program designs must withstand
significant variations in time (i.e., date of initiation, rates of
progress, completion milestones, etc.) and budget availability (both
total amount and annual appropriations).  Programs should be designed to
absorb reasonable schedule changes with minimal impact. Similarly,
robust programs should yield acceptable returns on investment over a
wide range of program size (i.e., they should be fairly elastic over
different scale levels). Programs that are designed as all or nothing
versus those that can be bought by the yard are especially brittle in
the face of inevitable uncertainties in the appropriations and
authorizations processes.

9. Institute Performance Measurement and Program Evaluation

Program designs should incorporate feedback mechanisms enabling managers
and policymakers to assess how well a program works in terms of both
implementation and achievement of program goals. Of course, program
designers first must sufficiently identify the program's goals and
specific performance objectives to measure outcomes.

Take, for example, the Customs Service's trade law violation detection
program. In fiscal 1991, U.S.  Customs did not detect about 84 percent
of the estimated trade law violations in imported cargo.[Endnote 7] Does
a detection rate of 16 percent indicate success? A strong design should
require benchmarks for success/failure and delineate when a program
should be redesigned, concluded because successful, or terminated for
failure.

Designers, in consultation with potential users of the performance data,
should monitor a range of key performance indicators.[Endnote 8] A wide
range of types of data could be collected and reported, including a
program's:

--inputs (e.g., funding and staff, beneficiary characteristics),

--workload,

--outputs (services/final products),

--outcomes of products or services (i.e., for a job training program,
the number ofjob placements that result in a year's employment),

--efficiency and/or productivity,

--beneficiary satisfaction,

--employee satisfaction, and

--service quality and timeliness.

A performance measurement system also requires identified data sources
and reliable collection and reporting methods. The value of the
information provided should be carefully weighed against the costs of
data collection, including the burden imposed on reporting entities
(such as state and local governments). Where practical, performance
reports should be designed to meet the diverse information needs (in
terms of both quantity and particular indicators presented) of the
various users of performance data (e.g., beneficiaries, line staff, and
program managers, senior agency officials, OMB, etc.). Nonetheless,
compromises among users may be unavoidable.

The new Government Performance and Results Act of 1993 will
institutionalize performance measurement throughout the federal
government. Key elements of this legislation include strategic plans,
performance plans, performance reports, managerial flexibility waivers,
and performance budgeting. This Act requires a variety of pilot projects
which could provide fruitful opportunities for the application of the
several program design criteria suggested here.

10. Build in Cessation Provisions

There is a common perception that some government programs and
organizations outlive their usefulness yet continue to exist without
rational purpose or tangible benefit. Government has often been
criticized as being totally incapable of abandoning programs, despite
success, failure, or lack of evidence of either. Many government
programs do not explicitly provide for their conclusion or termination
for any cause whatsoever. A program design which does not address the
ultimate fate of the program is at best incomplete, at worst self-
perpetuating. Although some programs have no expected expiration (e.g.,
Social Security), others do possess an ultimate end point (e.g.,
inoculation programs should cease when the disease is essentially
eliminated, as in the case of smallpox).  Determination should be made
at the onset as to what marks the success or failure of a program. When
these benchmarks are established, it will be easier to decide when to
redesign a program or just stop.

The Helium Fund Program was created in 1925 to ensure helium supplies
for blimps. It has accumulated a 176- year supply of helium along with a
$1.4 billion debt.  All original objectives (except paying for itself)
have been met, and private suppliers could meet all federal needs at
lower cost.[Endnote 9] The obvious question is who are the primary
beneficiaries of this program now that blimps are not critical to
national security or commerce? Moreover, the helium program lacks any
cessation provisions, an all too common failing in program design.

Sunset laws have a checkered history.[Endnote 10] While enjoying a surge
of popularity at the state level in the 1970s and early 1980s, sunset
laws have produced mixed results and many states are repealing
ineffective sunset legislation. In January 1993, a sunset bill (S. 186)
was reintroduced in the 103d Congress which would require formal
reauthorization of federal programs every ten years. Given the amount of
controversy around the applicability of sunset requirements to federal
agencies, the bill's future is uncertain. Perhaps more promising avenues
exist in strengthening reauthorization requirements by incorporating
rigorous performance measurements and enforcing appropriate discipline
in both the executive and legislative branches of government.

*************************************************
ENDNOTES

1. Five of these criteria are consistent with evaluation criteria
proposed by Senator Reid in his bill, S. 186, Spending and Control
Programs Evaluation Act of 1993.

2. Osborne, David, and Ted Gaebler, Reinventing Government (Reading, MA:
Addison-Wesley Publishing Company, Inc., 1992).

3. Salamon, Lester M., Beyond Privatization: The Tools of Government
Action (Washington, D.C.: Urban Institute Press, 1989).

4. See NPR Accompanying Reports, Department of Labor and Department of
Health and Human Services (Washington, D.C.: U.S. Government Printing
Office, September 1993).

5. "The great problem, as we understand it, is to make the difficulties
of implementation a part of the initial formulation of policy.
Implementation must not be conceived as a process that takes place
after, and independent of, the design of policy. Means and ends can be
brought into somewhat closer correspondence only by making each
partially dependent on the other.'' Pressman and Wildavsky,
Implementation (Berkeley, CA: University of California Press, 1973), p.
143.

6. See NPR Accompanying Reports, Executive Office of the President,
Streamlining Management Controls, and Strengthening the Partnership in
Intergovernmental Service Delivery .

7. U.S. General Accounting Office, High Risk Series:  Managing the
Customs Service, HR-93-14 (Washington, D.C.: U.S. General Accounting
Office, December 1992).

8. See Osborne and Gaebler, pp. 349-359, which describe several
approaches to performance measurement in the public sector. A variety of
others are referenced on p. 392.

9. "Helium Agency--Deterrent to Blimp Wars: Congress Must Decide Whether
1920s Program is Worth Saving,'' San Francisco Chronicle (May 21, 1993),
p. A5; and "Forget Oil: U.S. Rules Helium Market,'' Washington Post
(June 17, 1993), p. A17. See also NPR Accompanying Report, Department of
Interior (recommendation DOI13).

10. Kearney, Richard C., "Sunset: A Survey and Analysis of the State
Experience,'' Public Administration Review (January/February 1990), pp.
49-57; and Nice, David C., "Sunset Laws and Legislative Vetoes in the
States,'' State Government, vol. 58, no. 1 (Spring 1985), pp. 27-32.

*************************************************
Actions

What Can be Done to Develop and Promulgate Program Design?

Program design criteria are merely the foundation for more efficient and
effective federal programs. To establish a credible discipline of
program design, the criteria and the technology used in successful
design should be developed more fully throughout the federal government.
The following are recommendations for developing a formal discipline of
program design.