University of Vermont

Burlington, Vermont

Today's funding resources require specific and understandable evaluation methods to measure the impact of an extension program for a particular audience. Often the goals of a project can be measured by the number of participants involved and lessons learned. These measures are commonly used by traditional educational programs. Extension focuses on behavioral change as a desirable outcome of education. Government funded programs must succinctly state how they have made a difference in client behavior that goes beyond knowledge or attitudinal changes. To be accountable and test new methods of evaluation, the Women's Agricultural Network uses an eclectic, yet highly planned, approach to monitoring program impact. The approach clearly identifies the program objectives, indicators of effectiveness, and targeted goals for a period of five years.

The goal of the Women's Agricultural Network (WagN) is to enable more women to own/operate profitable farms and agriculturally related enterprises. In order to achieve this goal, we proposed: (1) education and guidance to develop a comprehensive business plan; (2) assistance in obtaining the necessary funding to grow and develop an agricultural business; (3) follow-up technical support to nuture the business; and (4) integration into the existing agricultural system. We proposed the following objectives:

Objective I

To increase the number of women engaged in agricultural endeavors the Network will provide the necessary information and education for participants to assess their personal level of commitment to the enterprise and to evaluate the agricultural alternatives available to them.

The Network will provide education on economically feasible and environmentally sound management practices that encourage sustainable agriculture and lead to increased productivity and profitability.

The Network will provide specific audiences with intensive education in business management in order to improve overall productivity. Participation will be limited and emphasis will be placed on individual assistance with management and production problems of individual operations.

The Network will build an effective network of federal, state and local partners that will provide women with a comprehensive continuum of services including but not limited to education, financial and technical assistance, and peer support.

The Network will develop a replicable model of outreach and assistance for other states to implement with various audiences.

The Network will strive to develop a long-term funding strategy that will enable it to continue on after the initial implementation period.

From these objectives, indicators of effectiveness were developed including: (1) numbers of individuals identified and contacted, (2) numbers of applicants for each program, (3) numbers of participants attending various program, (4) hours of technical assistance provided, (5) numbers of participants who have made business decisions based on information learned, and (6) number of women starting/expanding a business after accessing the Network. Thus, traditional measures such as contact numbers, program participation, and value (monetary and time) are made and reported.

Relevance of educational content to practice, skill development, fulfillment of educational needs and program objectives are also assessed. For example, year one impact measures included:

"Three hundred and fifty individuals will learn about the program opportunities through Orientation. One hundred individuals will attend "Getting Serious." Thirty individuals will participate in "Growing Places." WagN will provide 30 partial scholarships to Start Up for individuals starting agricultural related businesses. It is expected that 12 participants will begin/expand their businesses this year."

To capture the overall value of the program, a three pronged approach is used which involves constant revision of all analyses, traditional and non-traditional. Evaluation involves all potential players at monthly staff meetings, quarterly advisory council meetings and yearly retreats. Priority outcomes are believed to be best developed through processes that include at least three groups: (1) recipients of the resource, in this case network users, workshop/course attendees, and technical assistance clients, (2) coordinators and providers of the resource, in this case program employees and partners from other institutions and agencies, and (3) the larger context or community, including funders, citizens, legislators and regulators. For our complex systems' based project, the assistance to an individual is seen in the context of the participant's needs and its effect in the communities of location (geography), professional relationships (networking) and politics.

Preliminary program evaluation outcomes in this report focus on the traditional assessments. Recipient receptivity to and change generated by program activities include assessment of participants' backgrounds, information resources and aspirations and detailed course evaluations from two cycles of the course "Growing Places" are reported. Marketing efforts and advisory council development are also reported in quarterly performance reports. Further, the yearly evaluation of staff performance reviews includes individual and group goal assessments to generate further understanding of and commitment to evolving program goals.

In addition, case study strategies to follow the path of certain "bellwether" participants through an array of program resources are undergoing preliminary testing.

We have reached several conclusions based on the year-one results:

(1) Determine consistent high priority outcomes. Outcomes must be part of the design and program efforts should be proportional to the priority level of a specific outcome. Ongoing evaluation may revise the order of priority for different outcomes. We recognize that the "desirability" of an outcome is a value judgement. (2) Identify relevant criteria to judge and measure impact. (3) Focus efforts on multifactorial analyses, including participation level, immediate measures, measures over time, case studies and anecdotes. (4) Gather external information to make a case for "value added" results, including bench marking across programs.

Previous Page
Next Page

Return to Table of Contents
Return to Title Page