How to Design a Research Project in Environmental Economics

by David Glover
Economy and Environment Program for Southeast Asia

Preparing a proposal for a research project is a skill we tend to take for granted, although it is rarely taught in university. Written guides are limited to formats and application forms which briefly list the information requirements of a funding agency, but rarely explain why it important to provide that information.

A clear research proposal is important not only because it increases the chances of getting funding but because it is necessary to the design of an efficient project that will produce valid results. Many of the implementation problems that projects encounter could be avoided if the steps needed to carry out the project had been thought through and specified at the proposal stage.

This paper provides suggestions about how a thorough research proposal might be designed. The purpose is not to provide tips on how to fill out the forms. Instead it is to assist researchers in thinking through the steps needed to carry out a project for maximum impact with minimum effort, something that is in the best interests of the researcher as well as the funding agency.

Like most of us, the author has had no formal training in preparing research proposals. The comments made are based on fifteen years of experience in assessing research proposals as a staff member of IDRC. They were prepared for environmental economists and include examples from that field, but the principles are broadly applicable to other social sciences.

This paper is intended to help readers avoid some of the mistakes others have made and to make the research process a little bit easier. Good luck!

Step by Step

Research design is best approached by thinking through the research process in a sequential, step-by-step process. There will inevitably be some iteration and adjustments between steps, but the following is a useful starting point.

The principal steps are:

1. identify a problem
2. ask a clear question
3. limit the scope of the research
4. say what is known and not known about the problem
5. identify the skills and information needed to answer the question
6. select the methods needed to collect & analyse the data
7. explain the kind of results expected and how they will be presented
8. specify the timetable and budget

Before we even address Step 1, we should perhaps ask ourselves "What is the purpose of research?" If asked for a quick reply, many of us would probably say "To collect information". Certainly data gathering is an important part of most research. But it is not its purpose. The purpose of research is to answer a question. Preferably, it should be a question that someone (like a policy maker) needs an answer to in order to make a decision or to solve or avert a problem.

It follows from this that an essential initial step in designing a research project is to identify an important problem and pose a relevant and answerable research question. From this will flow the data requirements to answer the question, from that the methods needed to collect the data, and so on.

1. Identifying a Problem

For policy-oriented research, it is important to frame the problem as a policy maker would. A policy maker usually has the authority to deal with a particular aspect or manifestation of a problem; has only a limited range of instruments to deal with it; or faces pressures from a particular clientele to do something about it. For example, pollution is obviously a problem in most cities. But identifying "pollution" as a problem is not likely to lead to sharply-focused research useful to a policy maker. How would a policy maker frame the problem?

In the example above, it may be that the environmental agency is very concerned about emissions of toxic waste near a school. The problem is to reduce emissions and/or protect the population rapidly. In another case, the environmental agency is worried that the costs of reducing a widespread but non-toxic form of pollution will be excessively high, perhaps pushing firms out of business and creating unemployment. In that case, the problem is not to produce a rapid reduction in pollution but to identify least-cost instruments to achieve moderate reductions. In a third case, the agency may have been given the mandate and authority to deal with pollution, but no budget. In that case, the agency may be particularly interested in policy instruments that reduce pollution while simultaneously generating revenue for the agency (e.g. pollution taxes).

Research can also play a role in defining problems in ways that policy makers may not have conceived. For example, an agency facing a wide range of environmental problems may find itself overwhelmed and unable to set priorities. By assessing the cost of current damages from various kinds or sources of pollution, and the costs of addressing them, a research project could indicate where the highest cost-benefit ratios are to be found and help in ordering priorities.

The different approaches to economic analysis thus lend themselves to different purposes, e.g.:

* cost-benefit analysis - priority setting, choice of investments
* valuation of benefits - advocacy, sensitization, some priority setting
* cost-effectiveness analysis - identifying least-cost solutions.

The most important thing is that the problem should dictate the approach, not vice versa. The project must address an important problem, not apply a technique from its own sake. Many novice researchers, fresh from graduate school, become infatuated with techniques like linear programming, general equilibrium modelling, or valuation and then search for topics to apply them to. (Someone referred to this as the "law of the hammer", according to which a boy, given a hammer, finds everything worth pounding: not only nails but also Ming vases!)

2. Asking A Question

Having identified a real-world problem and framed it in terms relevant to a policy maker, the researcher must then pose an answerable research question or hypothesis. This essentially defines the objective of the project.

For an environmental economics project, this should be an economic question, i.e. one amenable to economic analysis. Economics tends to deal with issues like resource allocation, tradeoffs, and the difference between social and private costs and benefits.

Some of the characteristics of a good question are:

a) It should be a real question, answerable in some form (as opposed to pure description). Projects that seek to "describe the role of..." (...women in development, religion in politics, water in the ocean) tend not to be immediately useful.

b) Better still, it should be susceptible to a simple answer (yes or no; how much; which option should be selected). This also helps the researcher know when the project is finished; open-ended questions lead to open-ended projects.

c) An answer should be feasible with the resources available. It may be necessary to "cut the coat to fit the cloth", narrowing down the question to what the time and budget will allow.

Journalists are taught to answer simple questions in the first paragraph of a story: who, what, where, when, why and how. Researchers are not journalists, but these questions still provide good starting points for environmental economics projects. Interesting questions to pose include:

Who? Who will be affected by a policy or project? Who will pay the costs? Who will benefit?

What? What will it cost to solve the environmental problem? What will it cost not to solve it?

Why? Why do firms, households or governments do the things they do? What are the economic incentives that cause them to engage in environmentally damaging behaviour?

How? How can we change those incentives to change behaviour?

3. Limiting the Scope

One of the most common mistakes in designing a research project is a tendency to make the project too large and ambitious. If the objectives cannot be accomplished with time and resources available, then they should be restricted in some way. Possibilities for limiting the scope of a project include one or more of the following:

a) limit the geographical area

b) match the scope of the project to the mandate of the agency likely to use the results

c) focus on one resource or pollutant

c) focus on on-site effects (E.g. for soil erosion, look only at the effects on farmers suffering the erosion, excluding downstream effects like sedimentation of dams.)

d) conduct a financial rather than economic analysis (i.e. without shadow pricing). This will not provide a sufficient basis for decision-making but could be useful in explaining behaviour.

Estimating benefits can be particularly difficult, involving intangibles like health, aesthetics, biodiversity and so on. A thorough benefits estimation may take more resources than are available for the entire project and leave nothing for policy analysis. Alternatives to rigorous valuation of benefits include:

a) benefits transfer: taking and adjusting values of existing studies in other locations

b) cost-effectiveness analysis: Instead of estimating the value of costs and benefits (damages avoided) from various policies, a researcher might instead compare the costs of meeting a given standard by different means. This will not answer the fundamental question of whether the investment is truly worthwhile but it will allow the identification of the least-cost method of achieving the goal.

c) focus on major impacts: An ecosystem whose conversion is contemplated may include a dozen or more kinds of values (food production, storm protection, biodiversity, etc.). It may be that including only the most important two or three values will be enough to alter a decision. More information would be redundant.

d) "back of the envelope" estimates: It may be possible to do a careful original study of one aspect of the problem, and supplement that with quick and dirty estimates of other aspects.

Clearly, there are important trade-offs here. A narrowly focused project is likely to be more feasible and to produce more reliable estimates. A broader study is likely to attract more interest and have wider policy impact. At the extreme, a project that is too narrowly defined will no longer have an environmental dimension. For example, a cost-benefit analysis that includes only in-site effects, and is done in financial rather than economic terms will simply perpetuate the style of decision-making that led to environmental damage in the past. Limiting the scope of a project in a reasonable fashion calls for careful judgement; there are no simple rules. Whatever shortcuts are taken, these should be explicitly mentioned, not only in the research proposal, but in the final report as well.

4. Describing What is Known and Not Known

Most application forms for research grants include a section called "literature review". This is often seen as a burdensome formality, consisting of citations of two or three journal articles on vaguely similar topics.

In fact, it is very important to ascertain what information already exists on the research topic, in order to make the best possible use of existing data; avoid duplication; and get a clear idea of what resources are needed for new data collection.

Environmental economics is a relatively new field and it is unlikely that a previous study exactly like the one proposed has already been done. But there is usually something in the literature that can be useful to a new project. Possibilities include:

a) theoretical literature related to concepts or methods proposed in the study

b) empirical studies, either in the country to be studied, or in other countries. A good study done elsewhere may serve as a "prototype" and avoid the need to create a new research design from scratch.

c) raw data. It is especially important to know what physical data are available for economic analysis.

Finally, having explained what is known about this topic, what is not known? What is novel about the study proposed? What gap will it fill? The novelty of a new study could lie in the resource, pollutant or ecosystem to be studied; the method/s used (e.g. one valuation technique rather than another); or in the framing of the question (e.g. focusing on benefits of pollution control rather than costs).

Finally, having explained what is known about this topic, what is not known? What is novel about the study proposed? What gap will it fill? The novelty of a new study could lie in the resource, pollutant or ecosystem to be studied; the method/s used (e.g. one valuation technique rather than another); or in the framing of the question (e.g. focusing on benefits of pollution control rather than costs).

5. Identifying Skills and Information Needs

Rarely can a project in environmental economics be done solely by an economist. Collaborators with skills in physical sciences and other disciplines are usually needed. These people should be involved early in the design of the project, to help identify data needs and frame an answerable research question.

The research team should then identify the information (and only the information) needed to answer the research question. (Collecting too much information can be as big a problem as collecting too little.) Generally, one should not ask a question one does not need an answer to, except in early stages of reconnaissance or pre-testing a questionnaire.

6. Selecting Research Methods

This is one of the most important steps in designing a project and one that reviewers will devote the bulk of their attention to. The proposal should be as explicit as possible, identifying the research sites (or candidates for it); sample size for surveys; method for stratification of samples; frequency of surveys (e.g. one-shot or repeated); and so on. Pre-tests of questionnaires are highly recommended. If economic valuation methods are to be used, then the technique/s should be specified and an explanation offered as to how biases, gaps and double counting will be avoided. If more than one technique is to be used, it should be made clear whether the values from multiple techniques will be added to provide full estimates or compared for purposes of cross-checking. Where gender considerations are relevant, these should be addressed in the methodology (e.g. through disaggregation of data; assessment of the differential impact of policies and practices). Methods for data analysis (e.g econometric techniques) should also be specified and any biases or potential problems discussed.

It may be useful to think through the various steps in the research project and specify the method to be used for each step in table at the bottom of this page.

7. Disseminating Results

The design stage is not too soon to begin discussing the expected results with potential users. Involving them early may suggest research questions, facilitate access to data, and increase the likelihood of impact at the end. The proposal should describe any such discussions that have taken place and plans for dissemination, through seminars, publications, policy briefs, media, etc.

For further information on disseminating results effectively to policy makers, academics and journalists, see A Handbook for Disseminating Research Results in Environmental Economics, available from the EEPSEA Secretariat.

8. Planning the Timetable and Budget

The timetable should allow adequate time to carry the project through to completion. The project consists of more than data collection: time for pre-testing, data analysis and writeup is commonly underestimated.

The budget should specify the resources needed to carry out all the tasks specified above. It should be constructed from the "bottom up", not by dividing the total available into plausible proportions. For example, if fieldwork is involved, the budget for that item should be calculated by estimating the number of trips required, their duration and mileage, and so on.

Go With the Flow!

The sequential approach described above will minimize wasted time and avoid dead ends. Posing a clear research question will define information needs. Reviewing previous studies will avoid duplication and suggest promising approaches. Thinking through all the steps in the research process will make it easier to estimate a realistic budget. The result should be an efficient project that avoids major design problems and produces useful results.

Other References

EEPSEA, Guidelines for the Presentation of Research Proposals

EEPSEA, Handbook for Disseminating Research Results in Environmental Economics

D. Glover, Policy Researchers and Policy Makers: Never the Twain Shall Meet? Journal of Philippine Development, Number 38, Vol. XXI , Nos. 1&2, 1994

TABLE
PRELIMINARY LIST OF ENVIRONMENTAL DAMAGES TO BE ESTIMATED ARISING FROM THE MARCOPPER MINING DISASTER AND PROPOSED ECONOMIC VALUATION TECHNIQUES





Environmental Effect Economic Impact Proposed Valuation Technique




1.0 Damage to Common Property / Ecosystem (river and coastal)




1.1 Fisheries Decreased food production; foregone income from fish production Productivity change; substitution; travel cost

Subsistence

Commercial




1.2 Water


Household Increased consumption of alternative water sources;loss in recreation opportunities Substitution; travel cost; contingent valuation (CV)

Washing (clothes, utensils, etc.)

Bathing

Recreation (swimming, fishing)





Agriculture Reduced agricultural production Productivity change; replacement cost

Irrigation

Livestock (drinking, bathing)





Commercial Foregone income Productivity change; substitution

Laundry

Transport




1.3 Biodiversity (to be explored only) Loss of species, habitat Productivity change; CV




2.0 Damage to Private Property




2.1 Land (agricultural & residential) Loss in real estate and housing values; loss in livestock & poultry production Replacement cost; productivity change; hedonic pricing
2.2 Houses and other structures
2.3 Livestock and poultry
2.4 Others




3.0 Damage to Health (humans and animals)




3.1 Acute health effects Loss in earnings due to sickness; decreased meat and milk production Dose-response; cost of illness; productivity change; CV
3.2 Chronic health effects




4.0 Other Damages




4.1 Involuntary resettlement Cost of new infrastructure; social costs Replacement cost; CV

Source: E. Bennagen, "Estimation of the External Costs of Mining". Research Proposal, 1996

Economy and Environment Program for Southeast Asia