Evaluation of the Eastern Ontario Development Program (EODP)

Final report
Prepared by Evaluation Directorate
Federal Economic Development Agency for Southern Ontario
April 2016

Table of Contents

• Executive Summary
• 1.0 Introduction
  • 1.1 Objectives and Scope of the EODP Evaluation
  • 1.2 Outline of this Report
• 2.0 EODP Profile
  • 2.1 EODP Objective
  • 2.2 Background
  • 2.3 Previous Evaluations
  • 2.4 The EODP, 2011–12 to 2013–14
  • 2.5 Logic Model and Performance Measurement Strategy
  • 2.6 Stakeholders
  • 2.7 EODP Expenditures
• 3.0 Evaluation Methodology
  • 3.1 Evaluation Approach
  • 3.2 Data Collection Methods: Design and Application
  • 3.3 Evaluation Challenges and Limitations
• 4.0 Findings on Relevance
  • 4.1 Issue #1: Continued Need for Program
  • 4.2 Issue #2: Alignment with Government Priorities
  • 4.3 Issue #3: Alignment with Federal Roles and Responsibilities
• 5.0 Findings on Performance—Achievement of Expected Outcomes
  • 5.1 Achievement of the Immediate and Intermediate Outcomes
  • 5.2 Unintended Positive or Negative Outcomes
• 6.0 Demonstration of Efficiency and Economy
  • 6.1 Efficiency and Economy of the EODP Delivery Model in Producing Outputs and Outcomes
  • 6.2 Cost-Effectiveness in Achieving Expected Results
• 7.0 Conclusions and Recommendations
  • 7.1 Conclusions
  • 7.2 Recommendations
• Selected References
• Appendix A: 2011 EODP Evaluation: Recommendations, Management Responses and Actions Taken
• Appendix B: Eligible EODP (2011–14) Recipient Activities
• Appendix C: Program Management
• Appendix D: EODP Logic Model and Performance Indicators

List of Acronyms

BD

Business Development (business line)

CA

Contribution Agreement

CFDC

Community Futures Development Corporation

CFP

Community Futures Program

CI

Community Innovation (business line)

CP

Collaborative Projects (business line)

EAC

Evaluation Advisory Committee

EOCFDC

Eastern Ontario CFDC Network

EODF

Eastern Ontario Development Fund

EODP

Eastern Ontario Development Program

FedNor

Federal Economic Development Initiative for Northern Ontario

Gs&Cs

Grants and Contributions

GoC

Government of Canada

ICT

Information and Communications Technology

KI

Key Informant

MoSO

Minister of State’s Office

NFP

Not-for-Profit

PAA

Program Alignment Architecture

PMM

Performance Measurement Matrix

PMS

Performance Measurement Strategy

PSI

Post-secondary Institution

R&D

Research and Development

SD&G

United Counties of Stormont, Dundas and Glengarry

SODP

Southern Ontario Development Program

SOPI

Southern Ontario Prosperity Initiatives

SOPP

Southern Ontario Prosperity Program

SPD

FedDev Ontario’s Strategic Policy Branch

TBS

Treasury Board Secretariat

EVALUATION SUMMARY OF THEEASTERN ONTARIO DEVELOPMENT PROGRAM (EODP) – Infographic

PDF Version

320 KB, 1 page

EODP Promotes:

• Socioeconomic development in rural eastern Ontario
• Competitive and diversified regional economies
• Business and job opportunities
• Sustainable, self-reliant communities

Evaluation Period: April 1, 2011 to March 31, 2014

What the Evaluation Found

Success in Achieving Results and Exceeding Targets

• Undertook 2,050 projects
• Leveraged $73 million in additional funding ($2.71 for every dollar invested)
• Assisted 8,452 business
• Provided 3,113 workshops, training and counselling sessions for 12,528 employees

Success in:

• Building partnerships
• Supporting businesses and communities
• Creating and maintaining jobs
• Contributing to diversification and competitiveness

Performance

• Community Futures Development Corporations (CFDCs) delivering EODP works.
• Each CFDC has knowledge of the community’s needs and can respond to local priorities.

Recommendations

1. Plan collaborative projects, co-managed by more than one CFDC;
2. Ensure planning studies can be implemented;
3. Assess relative value of small vs. large projects;
4. Review CFDC funding approach;
5. Encourage CFDCs to share and leverage best practices and tools;
6. Use 2016 Census information to study socioeconomic situation in eastern Ontario;
7. Assess longer-term benefits for ultimate funding recipients; and
8. Assess EODP’s impact on FedDev Ontario’s ultimate outcomes.

For More Information

For more information, visit FedDev Ontario’s website for a complete list of evaluation studies, or contact FDO.Evaluation-Evaluation.FDO@feddevontario.gc.ca.

Executive Summary

An evaluation of the Eastern Ontario Development Program (EODP) was undertaken by the Federal Economic Development Agency of Southern Ontario (FedDev Ontario). EODP is a federal government grants and contributions program designed to promote socioeconomic development in rural eastern Ontario, leading to a competitive and diversified regional economy and contributing to the successful development of business and job opportunities, as well as sustainable, self-reliant communities.

The objective of the EODP evaluation was to determine the extent to which the EODP, in the three-year period from 2011–12 to 2013–14, was relevant, achieved its expected program outcomes, and demonstrated efficiency and economy. The evaluation was based on the Treasury Board Secretariat’s Policy on Evaluation (2009) and it’s accompanying Directive on the Evaluation Function, with a focus on the five core evaluation issues relating to relevance and performance.

The methodology used to address the five core evaluation issues in this calibrated evaluation of EODP involved the following data collection methods:

• Document and literature review;
• File review of performance, administrative (including business plans and contribution agreements) and financial data;
• Key informant interviews; and
• Data and cost analysis.

Overall, the evaluation methodology provided evidence that allowed FedDev Ontario to reach conclusions for all five core issues. However, there were limitations to the evaluation methodology. These limitations, described in the evaluation report, were taken into account during analysis and were recognized in the interpretation of the findings.

The evaluation was carried out by FedDev Ontario’s Evaluation Directorate during the period from October 2015 to January 2016. An Evaluation Advisory Committee (EAC) was established to provide advice to the evaluation project team at key points during the evaluation.

Profile of EODP

EODP has been an ongoing program since 2004 and was administered by Federal Economic Development Agency for Northern Ontario (FedNor) until 2009. Originally known as the Eastern Ontario Development Fund, it was launched as a one-year pilot program in response to program funding gaps and economic concerns expressed by regional stakeholders. The program was renamed EODP in 2006 and has been operating consistently since the original pilot. The current iteration of the program is scheduled to operate from 2014 to 2019.

FedDev Ontario assumed the overall management of EODP upon the Agency’s inception in 2009, and the 15 Community Futures Development Corporations (CFDCs) located across rural eastern Ontario have been responsible for third-party program delivery. CFDCs are independent, community‐based, legally incorporated not‐for‐profit (NFP) community development organizations. Each CFDC is managed by a board of directors made up of approximately 12 volunteers from the respective community. During 2011/12-2013/14, there were 37^{Footnote 03} CFDCs covering southern Ontario, of which 15 cover rural eastern Ontario.

During the 2011–12 to 2013–14 period, the CFDCs provided EODP grants and contributions (Gs&Cs) funding to eligible recipients, to support projects in the following three business lines:

• Business Development (BD)—Supported projects that would lead to growth of new and existing businesses within rural eastern Ontario communities. This included initiatives in skills development; youth retention and attraction; innovation; information and communications technology; and other business development.
• Community Innovation (CI)—Enabled community-led development projects that enhance and diversify local economies.
• Collaborative Projects (CP)—Encouraged projects eligible under the BD and CI business lines that involved investments from two or more CFDCs and benefited one or more communities.

Each CFDC developed a business plan and application to obtain EODP funding. In preparing their business plans, each CFDC consulted groups in their community, including county councils, municipalities, economic development and business groups, local politicians, provincial and other funding partners, and NFP organizations, among others. These consultations helped the CFDC to identify priorities that best responded to the needs of the community, within the program’s scope and activities as defined by FedDev Ontario. The business plan included the CFDC’s plans and targets for each of the BD, CI and CP business lines.

After reviewing the CFDCs’ business plans and applications, FedDev Ontario approved funding and signed a contribution agreement (CA) with each CFDC. The CFDCs then solicited applications for project funding in their communities, evaluated the applications, and decided which projects to fund. CFDCs put in place a project-level CA for each approved project, monitored the project’s progress, and approved and processed financial claims.

Program Relevance

The evaluation of program relevance was calibrated, given that the relevance of EODP had been well established in multiple program evaluations and that the program was renewed for the period starting April 1, 2014, and ending March 31, 2019.

Regarding EODP (2011–14),^{Footnote 04} interview respondents were generally very pleased with EODP and were very supportive of continuing the program, as it stimulates investment and plays a key role in the community. Further, respondents felt that while EODP resources were small in comparison to the size of the rural eastern Ontario economy, the program was extremely effective and was able to have an impact by helping to address major challenges and encouraging business and community development.

EODP was found to complement other federal and provincial government funding programs available to EODP project funding recipients. With their different emphases and funding, those programs served to fill the gap between the demand for funding in eastern Ontario and the limited funding available from EODP.

EODP (2011–14) was found to be consistent and fully aligned with government priorities, including FedDev Ontario’s Program Alignment Architecture and strategic outcome and federal government priorities and strategies, including the 2010 and 2011 federal budgets. EODP was also fully aligned with federal roles and responsibilities.

Program Performance—Achievement of Program Outcomes

EODP (2011–14) was generally successful in achieving results and exceeded almost all targets established for the program. With only a few exceptions, respondents reported that the BD and CI business lines were very successful. However, the CP business line, which involved investments from two or more CFDCs and benefited one or more communities, had mixed results, with some respondents indicating that it had been successful and others pointing to significant challenges.

Over the 2011–14 period, 2,050 EODP projects were undertaken: 1,387 (68 percent) were BD; 623 (30 percent) were CI; and 40 (2 percent) were CP. The EODP funding contributions of $27 million resulted in $73 million in additional funding leveraged from other sources, for total project expenditures of $100 million. Every dollar invested in EODP resulted in $2.71 in leveraged funds. This result was much greater than the targeted leverage of $1.68 per dollar invested, identified in CFDC business plans.

EODP helped create new businesses and assisted existing businesses that were struggling. In total, 8,452 businesses were assisted: 289 (3 percent) were created, 1,207 (14 percent) were expanded, and 6,956 (82 percent) were maintained. EODP funding resulted in 2,687 jobs being created and 14,646 jobs being maintained, for a total of 17,333 jobs through the supported projects. The majority of the job impacts were through the BD projects, where 11,023 (64 percent) of the total 17,333 were realized. The final result of 17,333 jobs created or maintained substantially exceeded the target of 4,500, which included an allowance for missing data. Details of EODP performance results are shown in the following table.

Performance Results for EODP (2011–2014)

Projects

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	Total Program
Number	1,387	623	40	2,050

Funding

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	Total Program
EODP funding contribution*	$14,279,989	$10,541,318	$2,187,027	$27,008,333
Leverage funding	$42,522,720	$25,954,056	$4,754,186	$73,230,962
Total project funding	$56,802,709	$36,495,373	$6,941,213	$100,239,295
Leverage ratio (leverage / EODP contribution)	2.98	2.46	2.17	2.71

Businesses

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	Total Program
Created	150	139	0	289
Expanded	805	384	18	1,207
Maintained	2,718	3,390	848	6,956
Total businesses	3,673	3,913	866	8,452
New markets and opportunities	1,352	1,482	229	3,063
Improved decision-making skills	1,308	1,451	293	3,052

Jobs

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	Total Program
Created	1,638	1,007	43	2,687
Maintained	9,386	4,177	1,083	14,646
Total jobs	11,023	5,184	1,126	17,333

Workshops, training and counselling

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	Total Program
Number of sessions	2,162	856	95	3,113
Trainees supported (including youth)	5,225	6,253	1,050	12,528
Youth trainees	3,551	3,818	59	7,428

Community economic development

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	Total Program
New economic sectors active	309	196	4	509
New staff & volunteers in economic development	641	1,254	36	1,931

Other areas

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	Total Program
Number of non-CFDC partnerships	1,147	1,725	215	3,087
Intern placements	297	20	7	324
Trade shows attended	239	55	10	304
Plans developed**	285	194	14	493

* The total EODP funding contribution for projects is 2.3 percent higher than the final expenditures shown in Exhibit 2.3 because final adjustments were made to expenditure figures after the program was completed.

** Business plans (activities) developed by CFDCs and their partners. The Contribution Agreements stipulate the development of such plans.

As an additional result of the EODP funding, 3,113 workshops, training and counselling sessions were undertaken in support of 12,528 trainees, including 7,428 young trainees. EODP funding also resulted in 509 new economic sectors becoming active across the 15 CFDC communities, and 1,931 new staff and volunteers becoming involved in community organizations dedicated to community economic development. EODP assistance resulted in an additional 3,087 non-CFDC partnerships, the placement of 324 interns, attendance at 304 trade shows, and the development of 493 plans (e.g., strategic plans, feasibility studies, business plans, community plans).

The performance information above indicates that EODP (2011–14) was successful in building partnerships, supporting businesses and communities, creating and maintaining jobs, and contributing to diversification and competitiveness.

Interview respondents viewed the CFDCs as being effective in contributing to rural economies and communities by supporting small business development; leveraging funding from other programs; and working with partners to promote economic stability, growth and diversification. Moreover, several respondents noted that given EODP’s limited budget, the program has been extremely effective in achieving results. While these results can be seen at the community level, it is difficult to assess the program’s impact on eastern Ontario given the size of the eastern Ontario economy.

Further, when interview respondents were asked if EODP had produced any unintended outcomes, the following comments were noted:

• The CFDCs benefited from delivering the EODP, as it increased their presence in the community and helped make their other programs more successful.
• EODP led to increased collaboration with other funding organizations in the community; it also reduced the fear of applying for government funding, as EODP is straightforward and decisions are made quickly.
• Leverage was much greater than anticipated, the program also benefited the supported organizations’ supply chains, and the improved performance indicators used in EODP (2011–14) led to more tangible results in the community.

In terms of challenges, interview respondents noted that CP initiatives were not as successful as they could have been; and the late start of EODP (2011–14) created some difficulties for the CFDCs, such as a number of NFPs not being able to implement some of their planned activities.

Program Performance—Efficiency and Economy

The EODP (2011–14) delivery model was found to be efficient and economical in producing outputs and progressing toward expected outcomes. The delivery model was considered appropriate, as it provides each CFDC, which has knowledge of the community’s needs, with the ability to respond to local priorities.

The demand for EODP funding was much higher than available funding, and program staff had a range of qualified applicants from which to select. Partnerships and leverage, including collaborating with the CFDCs as third-party delivery agents, were found to be instrumental in expanding FedDev Ontario's impact. The requirements that supported projects had to be completed by the end of the fiscal year (without the flexibility of carry-over from one year to the next) created problems and inefficiencies for the CFDCs. Further, it was noted that administrative processes and templates had been developed by each CFDC independently, and there was considerable variability among the CFDCs.

The third-party delivery model was found to be both efficient and effective. The cost of delivering an EODP dollar was found to compare favourably with the overall delivery cost of Gs&Cs programs by FedDev Ontario and by other regional development agencies. CFDC respondents reported a good working relationship with the FedDev Ontario project officers and were very satisfied with the services received. The delays in starting the program caused difficulties for the CFDCs (as CFDCs did require some time to develop and integrate new materials into their EODP roll-out plans and packages); however, they were successful in delivering the program.

There was a consensus among interview respondents that FedDev Ontario’s use of CFDCs to deliver EODP was the most cost-effective approach to achieving program results. It was further observed that there is no better, more cost-effective group of organizations that are lean, well established and fully connected in the community.

Recommendations

On the basis of the evaluation findings, the following recommendations are made for consideration by FedDev Ontario management:

Program Design

1. It is recommended that FedDev Ontario program management undertake the following:

1. Investigate whether there is a better approach to planning and managing Collaborative Projects (i.e., regional projects), which involve funding from two or more CFDCs and benefit one or more communities, to address the concerns identified in this evaluation.
2. Examine approaches to reduce the number of NFP planning studies for projects that do not have available funds to implement them.
3. Assess the relative value and balance of small projects versus large projects, given the decision to encourage larger, more strategic projects in 2013–14 and the fact that most businesses in eastern Ontario are small (micro) businesses.
4. Review the funding approach where each CFDC receiving equal contribution funding to determine whether funding should be allocated on a different basis that reflects the needs and priorities of the geographic areas covered.

Third-Party Delivery

2. It is recommended that FedDev Ontario program management undertake the following:

1. Revise the funding approach in which supported projects must be completed by the end of the fiscal year that they start in and allow multiyear projects.
2. Encourage the CFDCs to benefit from the various approaches (e.g., processes, templates, software) developed and used by the CFDCs across eastern Ontario for EODP management and administration (promotion, application processing, project evaluation and selection, etc.) by adopting the best practices and lessons learned, with the objective of improving efficiency. The Eastern Ontario CFDC Network could potentially lead this initiative.

Economic Research

3. It is recommended that FedDev Ontario management undertake a study of the socioeconomic situation in eastern Ontario to help determine the continuing need for a program similar to EODP as input to future program renewal. This study should be carried out after the 2016 Census information becomes available. This study could also include determining if there is merit in extending the program to areas with similar economic needs in southwestern Ontario.

Intermediate and Ultimate Outcomes

4. It is recommended that FedDev Ontario management include the following methodologies in future evaluations of EODP:

1. Assess the longer-term benefits to the project funding recipients. Also, investigate the role that EODP funding has in launching the supported projects; and determine the project funding recipients’ level of satisfaction and feedback in relation to the delivery of the program. This would likely include a survey of project funding recipients, interviews with CFDCs and project funding recipients, and case studies of supported projects.
2. Assess the program’s impact on the ultimate outcomes, which would involve undertaking special analyses using Statistics Canada data, subject to data availability.

1.0 Introduction

This report documents the evaluation of the Eastern Ontario Development Program (EODP) undertaken for the Federal Economic Development Agency of Southern Ontario (FedDev Ontario). This evaluation was undertaken to examine EODP’s program relevance and performance during the three-year period from 2011–12 to 2013–14.

EODP is a federal government grants and contributions program delivered by FedDev Ontario. EODP was designed to promote socioeconomic development in rural eastern Ontario, leading to a competitive and diversified regional economy and contributing to the successful development of business and job opportunities and sustainable, self-reliant communities.

FedDev Ontario is the federal government’s regional development agency responsible for southern Ontario. FedDev Ontario is headquartered in Waterloo and has offices in Toronto, Ottawa and Peterborough. EODP is managed out of the Peterborough office.

The evaluation was undertaken to meet the requirement of the Treasury Board Secretariat (TBS) Policy on Evaluation (2009) and to satisfy Financial Administration Act Section 42.1, which states that "every department shall conduct a review every five years of the relevance and effectiveness of each ongoing program for which it is responsible."^{Footnote 05} Previous evaluations of the EODP were undertaken in 2005, 2007 and 2011. The last evaluation was approved in March 2011, which requires this evaluation to be completed by March 2016.

The evaluation was carried out by FedDev Ontario’s Evaluation Directorate during the period from October 2015 to January 2016. An Evaluation Advisory Committee (EAC) was established to provide advice to the evaluation project team at key points during the evaluation.

1.1 Objectives and Scope of the EODP Evaluation

The objective of the EODP evaluation was to determine the extent to which the EODP in the 2011–12 to 2013–14 period was relevant, achieved or was on track to achieve its expected program outcomes, and demonstrated efficiency and economy. The evaluation was based on the TBS Policy on Evaluation and its accompanying Directive on the Evaluation Function (2009),^{Footnote 06} with a focus on five core evaluation issues relating to relevance and performance (effectiveness, efficiency and economy). For each of the five core evaluation issues, key evaluation questions were developed. The evaluation methodology is described in Section 3.0.

In Budget 2014, FedDev Ontario’s mandate was renewed for a further five years, and the renewal included a five-year extension of EODP beginning April 1, 2014. The EODP extension was included in the terms and conditions of FedDev Ontario’s Southern Ontario Prosperity Program as part of the Southern Ontario Prosperity Initiatives (SOPI) and is therefore included as part of the SOPI evaluation schedule. Significant changes were made to EODP at the time of its renewal, effective April 2014. Recognizing the above, the evaluation is focused on the three-year period from April 1, 2011, to March 31, 2014, and has been calibrated to balance the need for evaluation coverage and the responsibility to ensure effective stewardship of government resources.

1.2 Outline of this Report

This report contains a profile of EODP, the logic model and performance measurement strategy, and financial summary (Section 2.0); a description of the evaluation methodology (Section 3.0); findings on relevance (Section 4.0); findings on performance (Section 5.0); findings on efficiency and economy (Section 6.0); and conclusions and recommendations (Section 7.0).

2.0 EODP Profile

2.1 EODP Objective

The EODP’s objective is to promote socioeconomic development in rural eastern Ontario as a means to foster a competitive and diversified regional economy and in turn, the successful development of new business and employment opportunities in communities that are both sustainable and self-reliant.^{Footnote 07}

2.2 Background

In 2004, the Eastern Ontario Development Fund (EODF) was started as a one-year pilot program in response to program funding gaps and economic concerns expressed by regional stakeholders. The program has been continued since that time, with the name changing to EODP in 2006. EODP, as noted earlier, was most recently renewed for the 2014–19 period.

The following provides some highlights of EODP since its inception in 2004, EODF/EODP (2004–09)—The EODF program was announced with $10 million in funding for 2004–05, which was followed by successive annual renewals of $10 million. The name of the program was changed to EODP in July 2006. The program was delivered by the Federal Economic Development Initiative for Northern Ontario (FedNor) within Industry Canada until August 2009.

• EODP (2009–11)—Budget 2009 allocated $20 million to EODP over two years (2009–10 and 2010–11) as part of the creation of FedDev Ontario. EODP’s delivery was then undertaken by FedDev Ontario starting in August 2009.
• EODP (2011–14)—Budget 2010 provided $20 million for the renewal of EODP for an additional two years, which was subsequently amended to three years. This three-year period of EODP delivery from 2011–12 to 2013–14 is the subject of this evaluation.
• EODP (2014–19)—Budget 2013 provided $48 million in funding to extend EODP for an additional five-year period (2014–15 to 2018–19) as part of FedDev Ontario’s renewed mandate for an additional five years with a budget of $920 million.

2.3 Previous Evaluations

In the period since its inception in 2004, EODP has been evaluated three times:

• November 2005—A formative evaluation covering the first 10 months of the EODF assessed the design, delivery and implementation of the program. Overall, the program was deemed to be successful.
• March 2008—A summative evaluation of the program undertaken in 2007 found that the program was relevant in addressing the identified needs of rural eastern Ontario.
• March 2011—A summative evaluation undertaken in 2010–11 found that the program continued to be relevant and was instrumental in making investments in EODP priority areas.

In each evaluation, the program was deemed successful, and the use of the CFDCs as a primary delivery partner was seen as a major contributor to that success.

The March 2011 EODP evaluation provided recommendations on the design and delivery of the program, as well as the outcome performance data, collection tools and the use of performance information for the management of EODP. A summary of the recommendations in the 2011 evaluation and the Management Response and Action Plan are provided in Appendix A. This appendix also provides a description of the actions that have been taken by management and indicates that all recommendations have been addressed.

2.4 The EODP, 2011–12 to 2013–14

This subsection describes the EODP (2011–14) program, including how it was delivered through the Community Futures Development Corporations (CFDCs) to the project funding recipients; program management; and a breakdown of the projects funded.

2.4.1 Program Description

EODP’s three business lines, eligible project funding recipients, and funding eligibility are described below.

Business Lines

EODP (2011-14) provided grants and contributions support for projects in the following three business lines:

• Business Development (BD)—Supported projects that would lead to growth of new and existing businesses within rural eastern Ontario communities. This included initiatives in the following areas:
  • Skills development;
  • Youth retention and attraction;
  • Innovation;
  • Information and communications technology (ICT); and
  • Other business development.
• Community Innovation (CI)—Enabled community-led development projects that enhance and diversify local economies. Working in partnership with community stakeholders, CI was designed to optimize the economic and innovative capacity of rural eastern Ontario communities and develop opportunities for sustainable economic growth and employment.
• Collaborative Projects (CP)—Encouraged projects that involved investments from two or more CFDCs and benefited one or more communities. CP initiatives were expected to result in a greater number of strategic economic development initiatives and improved partnerships that would benefit eastern Ontario communities, including the strengthening of linkages between urban and rural areas. The objective of CP was to support initiatives that stimulate BD and CI opportunities by promoting socioeconomic development leading to a competitive and diversified regional economy in eastern Ontario. Eligible projects included those that would be eligible under the BD and CI business lines.

Detailed descriptions of eligible activities for projects contained in the signed contribution agreements (CAs) between FedDev Ontario and each CFDC are provided in Appendix B.

Eligible Project Funding Recipients

Recipients eligible for EODP project funding through the CFDCs were the following:

• Not-for-profit (NFP) organizations, including municipalities and municipal organizations, corporations, and community development organizations;
• Commercial enterprises, including individuals, corporations, partnerships, cooperatives and trusts; and
• Indigenous organizations.

Funding Eligibility

Eligibility for EODP project funding was as follows:

• For BD projects involving commercial enterprises, recipients were eligible to receive a non-repayable contribution not exceeding 50 percent of the total eligible project costs, up to a maximum of $100,000;
• For CI projects involving NFP organizations, recipients were eligible to receive a non-repayable contribution not exceeding 100 percent of the total eligible project costs, up to a maximum of $100,000; and
• For CP, recipients were eligible for the amounts stipulated above for BD and CI projects.

2.4.2 Program Delivery

FedDev Ontario was responsible for the overall management of EODP during 2011–12 to 2013–14 period. The program was delivered through 15 CFDCs located across rural eastern Ontario.

CFDCs are independent, community‐based, legally incorporated NFP community development organizations that were established to deliver the Community Futures Program (CFP) in a designated service area.^{Footnote 08} All CFDCs have corporate bylaws and have developed policies and procedures for administration of their general operations, personnel, and investment fund, as well as conflict of interest guidelines. Each CFDC is run by a board of directors made up of approximately 12 volunteers from the community. During 2011/12-2013/14, there were 37^{Footnote 09} CFDCs covering southern Ontario, of which 15 cover rural eastern Ontario.

The 15 eastern Ontario CFDCs deliver EODP as part of their program delivery. These CFDCs are the listed below:^{Footnote 10}

• 1000 Islands;
• Frontenac;
• Grenville;
• Haliburton;
• Kawartha Lakes;
• North & Central Hastings & South Algonquin;
• Northumberland;
• Peterborough;
• Prescott–Russell;
• Prince Edward/Lennox & Addington;
• Renfrew County;
• South Lake;
• Stormont, Dundas & Glengarry;
• Trenval; and
• Valley Heartland.

Exhibit 2.1 shows rural eastern Ontario in yellow, with the geographic areas covered by each of the 15 CFDCs. It should be noted that rural eastern Ontario excludes the cities of Ottawa and Kingston.^{Footnote 11}

Exhibit 2.1: Eastern Ontario and CFDC Delivery Areas

2.4.3 Program Management

The 15 CFDCs in eastern Ontario applied to FedDev Ontario for EODP (2011–14) program funding. FedDev Ontario approved the applications and signed CAs with the CFDCs setting out terms and conditions for third-party delivery. Through the CA, FedDev Ontario was responsible for the overall management of EODP, while the CFDCs were responsible for delivery of the program. FedDev Ontario’s EODP program officers worked with CFDCs to ensure locally based projects were effectively administered.

Each CFDC developed a business plan and application to obtain EODP funding. In preparing their business plans, each CFDC consulted with groups in their community to identify the priorities that best responded to the needs of the community.^{Footnote 12} The priorities were required to align with the program’s scope and activities as defined by FedDev Ontario.^{Footnote 13} The business plan included the CFDC’s plans and targets for each of the BD, CI and CP business lines.

FedDev Ontario reviewed the business plans and applications and approved funding, and then signed a CA with each CFDC. The CFDCs solicited applications for project funding in their communities, evaluated the applications and decided which projects to fund. CFDCs required a project CA for each approved project, monitored the project’s progress, and approved and processed financial claims. CFDCs were required to submit quarterly reports to FedDev Ontario describing activities undertaken, outcomes and impacts of the projects funded, as well as one success story. In addition, an annual performance report was required.

A more detailed description of FedDev Ontario’s and the CFDCs’ respective roles and responsibilities for delivering EODP (2011–14) is provided in Appendix C.

2.4.4 EODP-Funded Projects

During EODP (2011–14), the 15 CFDCs approved 2,050 projects for assistance from the 6,714 enquiries and 2,683 applications received. These figures indicate that, on average, 3.3 enquiries and 1.3 applications were received for each approved project. The total funding contribution for the 2,050 projects was $27.0 million, as shown in Exhibit 2.2.

Exhibit 2.2 also shows a breakdown of the EODP projects for the three business lines, with BD representing 1,387 (67.7 percent) of the total projects, CI representing 623 (30.4 percent) projects, and CP representing 40 (2.0 percent) of the total projects. In terms of EODP approved project funding, BD accounted for $14.3 million (52.9 percent), CI for $10.5 million (39.0 percent), and CP for $2.2 million (8.1 percent). In addition, the table provides a breakdown of the BD and CP business lines by their initiatives.

As the CP business line was made up of BD and CI projects, a separate table has been provided at the bottom of Exhibit 2.2 to show the summary figures for these two projects.

The exhibit also shows the average amount of funding for each of the business lines and their initiatives. These figures show that, on average, BD projects received $10,296 in EODP funding, while CI and CP received $16,920 and $54,676, respectively.

Exhibit 2.2: EODP Projects, 2011–12 to 2013–14

Project Breakdown (2011–14)	Projects Supported by EODP		Total EODP Project Funding Support		Average Funding per Project
	Number	Percent	Dollars	Percent	Dollars
Business Development	1,387	67.7%	$14,279,989	52.9%	$10,296
Skills Development	344	16.8%	$1,651,983	6.1%	$4,802
Youth Retention and Attraction	310	15.1%	$4,518,219	16.7%	$14,575
Innovation	90	4.4%	$1,717,161	6.4%	$19,080
ICT	224	10.9%	$1,877,298	7.0%	$8,381
Other Business Development Projects	419	20.4%	$4,515,329	16.7%	$10,776
Community Innovation	623	30.4%	$10,541,318	39.0%	$16,920
Collaborative Projects	40	2.0%	$2,187,027	8.1%	$54,676
Business Development	25	1.2%	$1,123,858	4.2%	$44,954
Community Innovation	15	0.7%	$1,063,169	3.9%	$70,878
Total	2,050	100.0%	$27,008,333	100.0%	$13,175
Business Development	1,412	68.9%	$15,403,846	57.0%	$10,909
Community Innovation	638	31.1%	$11,604,487	43.0%	$18,189
Total	2,050	100.0%	$27,008,333	100.0%	$13,175

2.5 Logic Model and Performance Measurement Strategy

A Performance Measurement Strategy (PMS) was prepared for EODP (2011–14) and finalized in January 2012. The PMS included an EODP logic model and a Performance Measurement Matrix (PMM) containing performance indicators for the overall EODP^{Footnote 14}.

2.5.1 Logic Model

The approved EODP logic model (2012) is provided in Exhibit D1 of Appendix D. It outlines the key activities of the program; the outputs that will result from those activities; and the immediate, intermediate and ultimate outcomes the program is intended to achieve.

2.5.2 Performance Measurement Strategy

In total, there were 11 outputs and outcomes and 45 performance indicators for the EODP (2011–14) program. EODP management implemented an in-depth data collection and progress reporting system to collect information on the performance indicators. The Performance Measurement Matrix (PMM) is provided in Exhibit D2 of Appendix D. The PMM also provides the source and frequency of performance information collection.

Performance information was collected by the 15 CFDCs from the project funding recipients and then summarized and provided to FedDev Ontario. Program management at FedDev Ontario checked and entered this information into a series of spreadsheets, which facilitated consolidation and stratification by CFDC, business line, initiative and key performance indicators.

The CFDCs provided target information for some of the performance indicators in the business plans that were submitted with their funding application. All CFDCs provided targets for the estimated number of projects to be undertaken and amount of leverage for the three business lines. In addition, most CFDCs provided targets for jobs created, or maintained, for the three business lines.

The quarterly reports provided performance information for almost all of the immediate outcomes, but for only some of the intermediate outcomes. The other indicator information would involve surveys or interviews with project funding recipients, which were not included in this evaluation.

Further, the measurement of the ultimate outcomes requires standardized methodology using Statistics Canada information such as the Census; the Labour Force Survey; and the Survey of Employment, Payrolls and Hours. These ultimate outcomes contribute to FedDev Ontario’s strategic outcome: a competitive southern Ontario economy. Since the program was completed on March 31, 2014, this evaluation was unable to provide an assessment of the program’s ultimate outcomes. This is a result of the two to five year period required for many projects to achieve their long term impacts as well as the time required for Statistics Canada to capture information regarding ultimate outcomes.

2.6 Stakeholders

FedDev Ontario worked with CFDCs to deliver EODP. Key stakeholders included the CFDCs, as well as the potential recipients of EODP funding. As noted above, the key stakeholder recipient groups that were eligible for EODP project funding were the following:

• NFP organizations, including municipalities and municipal organizations, corporations, and community development organizations;
• Commercial enterprises, including individuals, corporations, partnerships, cooperatives and trusts; and
• Indigenous organizations.

In addition to the recipient groups, the CFDCs worked with interested parties, such as municipal organizations, industry associations, volunteer groups, other CFDCs, departments and agencies of the Government of Ontario, and other federal departments and agencies (e.g., Business Development Bank of Canada).

2.7 EODP Expenditures

EODP (2011–14) was funded at $10 million per year for two years through Budget 2011 (March) from FedDev Ontario’s existing reference levels and was subsequently extended in 2013–14 for an additional $10 million, also from FedDev Ontario’s existing reference levels. In the first of the three years (i.e., 2011–12), the program was slow to launch. The program extension was announced in the late fall of 2011, and the 15 CFDCs submitted their applications and business plans near the end of 2011. The CAs between FedDev Ontario and the CFDCs were not signed until late February 2012.

The late signing of the CAs left only a short period until the end of March 2012 to undertake projects. Recognizing that it would not be possible to spend all the contribution funding identified for 2011–12 by March 31, FedDev Ontario adjusted the annual funding amounts over the three years so that fiscal 2011–12 received a smaller amount and 2012–13 and 2013–14 received larger amounts. Thus, the $30 million was expended over the three years.

The actual expenditures for the EODP during the three-year period from 2011 to 2014 are given in Exhibit 2.3.

Exhibit 2.3: EODP Final Expenditures, 2011–12 to 2013–14

EODP Actual Expenditures	2011–12	2012–13	2013–14	Total	Percent
FedDev Ontario—EODP Administration	$400,000	$400,000	$400,000	$1,200,000	4.0%
CFDCs—EODP Delivery	$2,370,638	$13,178,759	$13,145,183	$28,694,580	96.0%
Administration	$516,470	$889,608	$887,639	$2,293,717	7.7%
Contributions	$1,854,168	$12,289,151	$12,257,544	$26,400,863	88.3%
Total EODP	$2,770,638	$13,578,759	$13,545,183	$29,894,580	100.0%

Exhibit 2.3 shows that the contributions provided by the CFDCs for projects accounted for 88.3 percent of the total expenditures of $29.9 million, while the remaining balance included 7.7 percent for CFDCs’ administration costs for program delivery and 4.0 percent for FedDev Ontario EODP administration. Also, of the total of $28.7 million received by the CFDCs for EODP delivery, 7.7 percent ($2.3 million) was used by the CFDCs for administrative activities.

3.0 Evaluation Methodology

This section describes the methodology that was used in this calibrated evaluation of the Eastern Ontario Development Program (EODP), including the evaluation issues covered, data collection methods used, and the evaluation challenges and limitations.

3.1 Evaluation Approach

The EODP evaluation was based on the Treasury Board Secretariat (TBS) Directive on the Evaluation Function (2009),^{Footnote 15} which requires that evaluations assess all five of its core issues relating to relevance and performance (effectiveness, efficiency and economy) in order to address value for money. However, departments and agencies have the flexibility to determine the evaluation approach and level of evaluation effort in accordance with the program’s risks and characteristics and the quality of performance information available for each individual program.

The evaluation methodology was designed to align with the TBS Policy on Evaluation (2009), and as noted in Section 1.1, has been calibrated to adjust for the following factors:

• Budget 2014 announced a five-year extension of EODP beginning April 1, 2014.
• With the renewal of FedDev Ontario’s mandate effective April 1, 2014, EODP was included in the terms and conditions of FedDev Ontario’s Southern Ontario Prosperity Program as part of the Southern Ontario Prosperity Initiatives (SOPI) and therefore, is included as part of the SOPI evaluation schedule.
• Significant changes to the design of EODP came into effect on April 1, 2014.

Recognizing the above, this evaluation of EODP has focused on the years 2011–12 to 2013–14, capturing the period after the last evaluation and up to the start of the extension in 2014.

The evaluation was designed to address the five core issues relating to program relevance and performance required by the TBS Directive on the Evaluation Function. In assessing these five core issues, the evaluation focused on the key questions presented in Exhibit 3.1.

The evaluation design addressed the above evaluation issues and questions, recognizing and accounting for the following factors:

• In measuring performance (effectiveness), this evaluation primarily focused on immediate and intermediate outcomes. The quarterly reports provided performance information for almost all of the immediate outcomes but only some of the intermediate outcomes. The other indicator information would involve undertaking surveys or interviews with project funding recipients, which were not included in this evaluation. However, some qualitative information on the performance indicators was obtained through the CFDC interviews.

  Exhibit 3.1: Key Evaluation Questions for the EODP Evaluation, by Core Issue

  Relevance

  Issue #1: Continued need for program	1.1 Is there a continued need for the EODP?
  	1.2 Does the EODP complement, duplicate or overlap other government programs? Other private sector services?
  Issue #2: Alignment with government priorities	2.1 To what extent is the EODP consistent with government priorities: FedDev Ontario’s Program Alignment Architecture and strategic outcome? Federal priorities and strategies?
  Issue #3: Alignment with federal roles and responsibilities	3. To what extent is the EODP aligned with the federal government's activities, roles and responsibilities?

  Performance (effectiveness, efficiency and economy)

  Issue #4: Achievement of expected outcomes	4.1 To what extent have the expected immediate, intermediate and ultimate outcomes been achieved as a result of the EODP?
  	4.2 Has the EODP produced unintended positive or negative outcomes?
  Issue #5: Demonstration of efficiency and economy	5.1 To what extent is the EODP delivery model efficient in producing outputs and progressing toward expected outcomes?
  	5.2 Is there a more cost-effective way of achieving the expected results, taking into consideration alternative delivery mechanisms, best practices and lesson learned?

• The measurement of the ultimate outcomes provided in EODP Performance Measurement Strategy (see Section 2.5 above) requires that standardized methodology be used across all FedDev Ontario programs to measure FedDev Ontario’s impact on the southern Ontario economy. As the program was completed on March 31, 2014, and it takes two to five years after that for project outcomes to mature and then be reflected in the Statistics Canada databases that provide information on how the program contributed to the ultimate outcomes, this evaluation was unable to undertake an assessment of the ultimate outcomes.
• Performance had to be examined from the viewpoint of efficiency and economy in relation to the costs of the delivery models used and the achievement of outputs and results.
• Where appropriate, the evaluation design took advantage of, and built on, previous reports on EODP, as described in Exhibit 3.2.

  Exhibit 3.2: Relevant Documents

  Data Sources	Description	Year
  EODP Draft Evaluation Methodology Report	Described in draft form, the calibrated methodology being implemented in this evaluation	2015
  EODP Performance Measurement Strategy (PMS)	Described the PMS for EODP (2011–14), including the Logic Model, outputs and outcomes, and the Performance Measurement Matrix	2012
  Evaluation of the Eastern Ontario Development Program (EODP)	Provided the results of the evaluation for the EODP prior to EODP (2011–14)	2011
  Evaluation of the Community Futures Program (CFP)	Reported on the relevance and performance of the CFP in southern Ontario, which is delivered by CFDCs, the same organizations that deliver EODP in rural eastern Ontario	2014

3.2 Data Collection Methods: Design and Application

Considering the overall approach described above in Section 3.1, and recognizing that a calibrated approach was required, the data collection methods used in this evaluation and their application to the evaluation issues and questions are described below.

3.2.1 Application of Data Collection Methods to the Evaluation Issues and Questions

The selection of data collection methods for the evaluation was based on the most efficient means of rigorously addressing the evaluation issues and questions within the calibrated scope, budget and timeframe for the evaluation.

The following data collection methods were used in this evaluation:

• Document and literature review;
• Review of performance, administrative (including business plans and contribution agreements) and financial data;
• Key informant (KI) interviews; and
• Data and cost analyses.

The application of the above data collection methods to the evaluation issues and questions is shown in Exhibit 3.3.

Exhibit 3.3: Data Methods for Key Evaluation Issues and Questions

Relevance

Key Evaluation Issues and Questions		Document and Literature Review	Review of Performance, Admin. & Financial Data	Key Informant Interviews	Data and cost analyses
Issue #1: Continued need for program	1.1 Is there a continued need for the EODP?
	1.2 Does the EODP complement, duplicate or overlap other government programs? Other private sector services?
Issue #2: Alignment with government priorities	2.1 To what extent is the EODP consistent with government priorities: FedDev Ontario’s PAA and strategic outcome? Federal priorities and strategies?
Issue #3: Alignment with federal roles and responsibilities	3. To what extent is the EODP aligned with the federal government's activities, roles and responsibilities?

Performance (effectiveness, efficiency and economy)

Key Evaluation Issues and Questions		Document and Literature Review	Review of Performance, Admin. & Financial Data	Key Informant Interviews	Data and cost analyses
Issue #4: Achievement of expected outcomes	4.1 To what extent have the expected immediate, intermediate and ultimate outcomes been achieved as a result of the EODP?
	4.2 Has the EODP produced unintended positive or negative outcomes?
Issue #5: Demonstration of efficiency and economy	5.1 To what extent is the EODP delivery model efficient in producing outputs and progressing toward expected outcomes?
	5.2 Is there a more cost-effective way of achieving the expected results, taking into consideration alternative delivery mechanisms, best practices and lesson learned?

3.2.2 Description of the Data Collection Methods

The data collection methods used in the evaluation are described below.

Document and Literature Review

A review of relevant documents was carried out to obtain a good understanding of EODP, including its business lines, rationale and historical context. The review included program descriptions and background information as follows:

• Corporate and accountability documents—Included FedDev Ontario Departmental Performance Reports, Reports on Plans and Priorities, the EODP Performance Measurement Strategy (2012), and other documentation that provided information on FedDev Ontario and Government of Canada priorities (e.g., budgets, Speech from the Throne);
• Evaluations and reviews—Included key findings from the most recent evaluation of EODP(completed in March 2011) and current status of the implementation of the Management Response and Action Plan;
• Program materials and operational documents—Included program application requirements, applications and business plans, contribution agreements (CAs), transition and briefing notes, presentations, and reporting templates and tools; and
• Literature and research on economic development—Included research to identify studies examining the economy of rural eastern Ontario in order to determine the need for regional development programs in eastern Ontario.

File Review of Performance, Administrative and Financial Data

Documents containing performance, administrative and financial data on EODP that are collected and reported by FedDev Ontario programs and Finance were reviewed. Performance data for the program, collected from the CFDCs, was captured internally by FedDev Ontario in a series of spreadsheets. These spreadsheets facilitated consolidation and stratification by CFDC, business line, initiative and key performance indicators.

Key Informant Interviews

Interviews with key informants (KIs) were a primary information source for the evaluation. In total, 21 interviews were undertaken with the following groups:

• FedDev Ontario EODP staff (4);
• CFDC management and others (9), consisting of:
  • CFDC managers (5) that are responsible for managing all aspects of their CFDC, including involvement in the EODP, and report to their CFDC board of directors,
  • EODP coordinators (3) that are directly responsible for the delivery of EODP and report to the CFDC manager,
  • CFDC director (1) serves on CFDC boards of directors; and
  • Other (8), consisting of partner organizations (e.g., Province of Ontario, Business Development Bank of Canada) and community and academic representatives.

The KIs were identified in consultation with FedDev Ontario’s EODP management and the Evaluation Directorate. For each of the targeted groups, semi-structured interview guides were developed to address the key evaluation questions in Exhibit 3.1. All the interviews were conducted by telephone. Interview findings were captured and entered into an electronic database and were summarized according to the evaluation issue, indicator and respondent type.

Data and cost analyses

A number of data and cost analyses were undertaken in support of the evaluation, in particular for issue #5 on demonstration of efficiency and economy. Several of these analyses examined the delivery of the program, and have been provided in Section 6.0.

3.3 Evaluation Challenges and Limitations

In undertaking this calibrated evaluation, the evaluation project team identified the following challenges and limitations that are discussed below.

3.3.1 Challenges

EODP was extended in 2014, and changes to the program were implemented—In selecting KIs, the team made an effort to identify individuals with knowledge of the program during the evaluation period from 2011–12 to 2013–14. FedDev Ontario and CFDC personnel were generally familiar with EODP (2011–14) and were able to respond to questions about EODP during the 2011–12 to 2013–14 period. However, many of the respondents also referred to the changes made in the subsequent EODP (2014–19) when commenting on EODP (2011–14). Other KIs had a general familiarity with the program but were not always able to distinguish between the specifics of each EODP.

3.3.2 Limitations

Coverage of government, CFDCs and program funding recipient stakeholders—The 21 KI interviews provided good coverage of government and the CFDCs that directly received funding from FedDev Ontario. However, the KI interview methodology is limited by the fact that all findings reflect the views and opinions of only the individuals who were interviewed, who did provide insightful information on the program delivery and benefits. However, project funding recipients were not included in this calibrated evaluation.

Timely and accurate information—To measure the performance of the program, FedDev Ontario depends on project funding recipients providing the information on their projects. This information is collected by the CFDCs and then passed on to FedDev Ontario through the quarterly reports. While the project funding recipients have provided the performance information as required by the CAs, FedDev Ontario relies on the CFDCs to review the collected information before it is forwarded to FedDev Ontario. FedDev Ontario has only limited capacity to verify the performance information provided by the project funding recipients, beyond a check for reasonableness by program management. However, this approach to collecting information on performance and results is the only cost-effective way of obtaining this information and is extensively used in similar evaluations.

Measurement of impacts—The performance information collected by FedDev Ontario was complete up to March 31, 2014; however, many of the project impacts would have occurred after that date. While some of the projects would achieve benefits in the short term (one to two years) after project completion, many would be expected to achieve benefits in the medium term (three to five years) or the long term (beyond 5 years). Further, the measurement of some of the intermediate outcome indicators requires follow-up with the project funding recipients to obtain quantitative and qualitative information on the impacts of the funding; however, a survey of project recipients was not included in this evaluation. Therefore, the information from the performance reports as of March 31, 2014, provides a limited picture of the benefits, and the indicators measured through the quarterly reports likely underestimate the overall impacts of the funded projects.

It takes two to five years after the program is completed, as noted above, for the projects to achieve their full benefits and for the Statistics Canada databases to provide information on how the program contributed to the ultimate outcomes. An assessment of the ultimate outcomes was not feasible within the scope and time frame of this evaluation.

Impact on the southern Ontario economy—The impact of the EODP contributions is relatively small when compared to the economy in the region. Moreover, other investment programs are also active in the region. It is therefore difficult to determine the extent to which EODP has contributed to FedDev Ontario’s strategic outcome: a competitive southern Ontario economy.

4.0 Findings on Relevance

This section provides the evaluation findings for relevance. These findings have been structured on the basis of the three core relevance issues described in Section 3.1.

• Issue #1: Continued need for program
• Issue #2: Alignment with government priorities
• Issue #3: Alignment with federal roles and responsibilities

4.1 Issue #1: Continued Need for Program

4.1.1 Was There a Continued Need for a Program Similar to EODP?

The relevance of the Eastern Ontario Development Program (EODP) has been well established since its inception in 2004. Past evaluations of the program and the government’s renewal of the EODP for the 2014–19 period confirm its relevance.

The KIs noted that there have been economic challenges in rural eastern Ontario since the start of EODP (formerly known as the Eastern Ontario Development Fund [EODF]) in 2004 and that the recession resulting from the 2008–09 financial crisis hit rural eastern Ontario hard, resulting in more systemic challenges for the area.

The challenges that the KIs mentioned included layoffs and plant closings, increased unemployment, outmigration of youth (those who stayed lacked skills, and those who left didn’t come back), lack of opportunities for young people, underemployment, an aging workforce, a shortage of skilled labour, reduced access to capital (inability to fund small businesses), low productivity, no synergy from having business clusters or infrastructure, lack of diversification (traditional sectors of agriculture and tourism are low paying and seasonal), inability to attract businesses, limited access to high-speed Internet (broadband), and low web presence (website and e-commerce).

In addition, the high value of the Canadian dollar versus that of the US dollar^{Footnote 16} leading up to 2011 affected competitiveness and contributed to the loss of manufacturing and reduced tourism. US trade protection was also noted as a factor. Another challenge was the effect of lower commercial and industrial activity on the tax base of municipalities, which made it difficult for them to maintain infrastructure and provide services. Transportation infrastructure, both road and rail, was also noted as a challenge.

EODP (2011–14) was designed to help address the challenges noted above (see Section 2.0). Within the parameters of EODP, the 15 Community Futures Development Corporations (CFDCs) had the flexibility to target the assistance to meet the key challenges in their communities. Respondents felt that EODP had done a good job addressing the challenges, considering its size and resources; however, the resources did not come close to meeting demand. It was recognized that EODP could not address these challenges on its own, but had made a difference by contributing to the solutions. For example, while EODP (2011–14) did not have the resources to improve broadband coverage, it assisted businesses by supporting websites, e-commerce, and IT solutions. For example, it was observed that EODP "plants the seed" and helps mitigate risks of new initiatives. EODP may support a feasibility study that results in a major investment several years later.

Respondents were generally very pleased with EODP and were very supportive of continuing the program, as it plays a key role in the community. EODP stimulates investment in the community. Hundreds of youth would not have found work in their home towns. Worker skill shortages would not have been addressed. Economic development takes time, as it involves multiple years and multiple efforts, and EODP has been instrumental in keeping the momentum going. Several CFDCs commented that EODP strengthens the role of the federal government in rural eastern Ontario. An "Other" respondent stated that EODP is critical to rural eastern Ontario, particularly for smaller businesses.

In summary, the area benefits from EODP, as it encourages growth of new and existing businesses and enables community-led development projects that enhance and diversify local economies within rural eastern Ontario communities.

4.1.2 Did EODP Complement, Duplicate or Overlap Other Government Programs or Other Private Sector Services?

EODP is flexible, has broad application within its scope, and is available to a wide group of applicants. Other programs are more targeted or restricted (e.g., hi-tech or agriculture; larger firms) and have specialized requirements. A respondent commented that there is never money to develop strategy for economic development, which is supported by EODP, and this helps drive the other programs. Further, EODP focuses on small (micro) businesses with fewer than 10 employees, which accounts for almost all the businesses in rural eastern Ontario. However, EODP also works with larger businesses of up to 300 employees, of which there are very few.

All respondents commented that EODP was complementary to other federal and provincial government programs that may be available to its beneficiaries. In terms of complementarity, one approach is to nurture the small businesses forward with EODP, and then when the business is larger, support them with other programs. For example, Mariposa Dairy Ltd., received $63,827 under EODP for the period of this evaluation and has been awarded another $1M through the Southern Ontario Prosperity Program (SOPI) from June 26, 2014 to December 31, 2016. In addition, there are examples where EODP funds certain components of given projects, with complementary programs funding other components.

Respondents commented that EODP doesn’t duplicate other funds and that there is an absence of federal programs in eastern Ontario targeting the same beneficiaries. Further, EODP complements provincial programs, which work well together. For example, EODP does capacity support, such as business planning and skills development, which are critical precursors to the Province of Ontario funding capital (machinery and equipment) through its own Eastern Ontario Development Fund, which is designed for much larger projects. The province views EODP as being very important and complementary.

EODP is also complementary to the Community Futures Program (CFP)^{Footnote 17}, a national loan program in southern Ontario, which is administered by FedDev Ontario and delivered by the CFDCs to the same recipients. Where appropriate, in delivering CFP the CFDCs may reduce the loan and include a grant component from EODP.

CFDCs coordinate efforts closely within their respective geographic areas. They are aware of their key community stakeholders, and those stakeholders are aware of the CFDC’s role. They work in collaboration with other funders, including the province and other FedDev Ontario program funds, to try to determine how to best leverage government programs to support companies. The organizations collaborate on larger projects. CFDCs are often co-located with other programs in municipal buildings, resulting in fewer stops for proponents, increased synergies, reduced costs, etc. Triaging applicants was mentioned by several respondents as the approach used to determine how best to support companies with programs and referrals. This involves identifying applicants’ core needs and then matching them with the appropriate programs.

All respondents agreed that that EODP complemented other federal and provincial government funding programs where they were available to the project funding recipients. The existence of other programs was not viewed as a concern, as other programs had a different emphasis and the demand for funding in eastern Ontario was much greater than the limited funding available.

4.2 Issue #2: Alignment with Government Priorities

Budget 2010 announced that funding would be provided for EODP for 2011–12 and 2012–13 as part of "Support for Communities." The Budget’s focus was on "helping Canadian firms create jobs, modernize their operations and better compete globally." The $20 million over two years was again mentioned in Budget 2011 and in Budget 2012. Funding for EODP was subsequently amended to include funding for 2013–14. Budget 2013 then extended EODP for five years to "continue to promote business development, job creation and self-reliant communities in rural eastern Ontario" as part of the $920 million, five-year renewal of FedDev Ontario’s mandate as of April 1, 2014.

EODP was designed to support the community economic development program activity in FedDev Ontario’s Program Alignment Architecture. In turn, the community economic development program activity supported FedDev Ontario’s strategic outcome ("a competitive southern Ontario economy") and contributed to the Government of Canada’s outcome ("strong economic growth").^{Footnote 18}

EODP was therefore consistent and fully aligned with government priorities and strategies.

4.3 Issue #3: Alignment with Federal Roles and Responsibilities

FedDev Ontario, as a regional development agency and one of the federal government's representatives, was aligned with federal government priorities through EODP to encourage growth of new and existing businesses; and to enable community-led development projects that enhance and diversify local economies within rural eastern Ontario communities. EODP’s program activities were designed to contribute to the diversification and competitiveness of the rural eastern Ontario economy; economic stability and growth and job creation; and sustainable, self-reliant communities. Respondents considered EODP to be fully aligned with federal roles and responsibilities and to be a formal and explicit match with government policy.

Past Speeches from the Throne have placed emphasis on economic growth and stability, specifically, to support traditional industries, create jobs, and minimize loss of employment. The Speech (2011) noted that "jobs and growth will remain our government’s top priority."^{Footnote 19}

5.0 Findings on Performance—Achievement of Expected Outcomes

This section provides EODP evaluation findings for achievement of expected outcomes. The findings have been structured around the two key questions provided in Exhibit 3.1.

• 4.1 To what extent have the expected immediate, intermediate and ultimate outcomes been achieved as a result of the EODP?
• 4.2 Did the EODP produce unintended positive or negative outcomes?

5.1 Achievement of the Immediate and Intermediate Outcomes

This subsection addresses the question "To what extent have the immediate and intermediate outcomes been achieved as a result of EODP?" An assessment of the ultimate outcomes was not feasible within the scope and timeframe of the evaluation.

Program outputs and outcomes result from the completion of EODP projects, which contribute to the achievement of the program objectives. The linkage between the program activities, outputs and outcomes is shown in the EODP logic model in Appendix D.

The achievement of immediate and intermediate outcomes is summarized in Section 5.1.1. The assessment of these outcomes is based on FedDev Ontario’s comprehensive performance information, key informant (KI) interviews, file review and document review. Success stories, prepared by FedDev Ontario, have been used to illustrate how EODP contributed to project successes. Activities and outputs are included in the discussion, as they are directly linked to achieving the outcomes.

5.1.1 Overall Results

Overall, EODP was successful in achieving results and exceeding almost all targets established for the program. The business lines Business Development (BD) and Community Innovation (CI) were found by respondents to have been very successful with some reservations (section 3.3). Collaborative Projects (CP), which involved multiple Community Futures Development Corporations (CFDCs) and benefited one or more communities, had a mixed result, with some respondents feeling that CP had been successful and others feeling that it had significant challenges.

5.1.2 Performance Results

A summary of the performance results for EODP (2011–14) for the three business lines (BD, CI, CP) is provided in Exhibit 5.1. This information was compiled from the quarterly performance reports submitted by the 15 CFDCs to FedDev Ontario throughout the program.

Performance Results for EODP (2011–14)

Projects

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	TotalProgram
Number	1,387	623	40	2,050

Funding

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	TotalProgram
EODP funding contribution*	$14,279,989	$10,541,318	$2,187,027	$27,008,333
Leverage funding	$42,522,720	$25,954,056	$4,754,186	$73,230,962
Total project funding	$56,802,709	$36,495,373	$6,941,213	$100,239,295
Leverage ratio (leverage / EODP contribution)	2.98	2.46	2.17	2.71

Businesses

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	TotalProgram
Created	150	139	0	289
Expanded	805	384	18	1,207
Maintained	2,718	3,390	848	6,956
Total businesses	3,673	3,913	866	8,452
New markets and opportunities	1,352	1,482	229	3,063
Improved decision-making skills	1,308	1,451	293	3,052

Jobs

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	TotalProgram
Created	1,638	1,007	43	2,687
Maintained	9,386	4,177	1,083	14,646
Total jobs	11,023	5,184	1,126	17,333

Workshops, training and counselling

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	TotalProgram
Number of sessions	2,162	856	95	3,113
Trainees supported (including youth)	5,225	6,253	1,050	12,528
Youth trainees	3,551	3,818	59	7,428

Community economic development

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	TotalProgram
New economic sectors active	309	196	4	509
New staff & volunteers in economic development	641	1,254	36	1,931

Other areas

Results	Business Development (BD)	Community Innovation (CI)	Collaborative Projects (CP)	TotalProgram
Number of non-CFDC partnerships	1,147	1,725	215	3,087
Intern placements	297	20	7	324
Trade shows attended	239	55	10	304
Plans developed	285	194	14	493

The figures show that for the 2,050 projects undertaken, 1,387 (68 percent) were BD; 623 (30 percent) were CI; and 40 (2 percent) were CP. The performance results in Exhibit 5.1 are summarized below:

• Leverage—The EODP funding contributions of $27 million leveraged additional funding of $73 million from other sources, for total project expenditures of $100 million. The leveraged funding was therefore 2.71 times the EODP contributions. The leveraging for BD was higher than for CI, which in turn was higher than for CP.
• The actual leveraging considerably exceeded the target for the program, which was the aggregate of the targets provided by the CFDCs in their business plans. The target for overall leveraged funding was an additional $44 million (vs. $73 million achieved), or 1.68 (vs. 2.71 achieved) times the EODP contributions. The leveraging factors achieved for BD (2.98) and CI (2.46) exceeded their targets of 1.54 and 1.69, respectively, while CP at 2.17 was lower than the targeted 2.45.
• Business assistance—EODP helped create new businesses and assisted existing businesses that were struggling. In total, 8,452 businesses were assisted: 289 that were created^{Footnote 20} (3 percent); 1,207 that were expanded (14 percent); and 6,956 (82 percent) that were maintained. In addition, 3,063 businesses were assisted with identification of new or external markets and business opportunities, and a further 3,052 businesses were helped to develop their decision-making skills.
• Job impacts—Through EODP, the CFDCs reported that the projects receiving funding assistance resulted in 2,687 jobs being created and a further 14,646 jobs being maintained, for a total of 17,333. The majority of the job impacts were through the BD projects, with 11,023 (64 percent) of the total 17,333 jobs realized.
• The target for the number of jobs estimated for the program by the CFDCs when submitting their business plans was close to 4,500. The final result of 17,333 substantially exceeded this target.
• Workshops, training and counselling—The program ran 3,113 sessions, during which 12,528 trainees were supported, of which 7,428 were youth.
• Development—As a result of the projects, 509 new economic sectors became active across the 15 CFDC communities, and 1,931 new staff and volunteers became involved in community organizations dedicated to community economic development.
  • Other areas—EODP assistance resulted in an additional 3,087 non-CFDC partnerships (partnerships developed with local municipal Economic Development Offices, post-secondary institutions, economic development organizations, business-led organizations), the placement of 324 interns, attendance at 304 trade shows, and the development of 493 plans (e.g., strategic plans, feasibility studies, business plans, community plans).

The quarterly reports provided information on the performance indicators for almost all of the immediate outcomes, but only some of the intermediate outcomes (e.g. leveraged amounts and job impacts).

5.1.3 Respondent Observations

The following comments were provided by the respondents interviewed as part of the evaluation. The comments provide some insight into the program, but these comments may not be inferable, given that not all CFDCs were included in the interview program.

Business Development

BD was generally viewed as being successful or very successful. The following observations were made by the respondents on the BD initiatives:

• Skills development was found to be very successful and worked well. One respondent noted that it seemed to work better with larger businesses, as small (micro) businesses didn’t seem interested.
• Youth retention and attraction was also considered to be very successful and effective by the majority of respondents, although some did not achieve the same level of success. One respondent observed that youth retention and attraction was successful, particularly the intern program (up to a year) to build skills and help offset costs to business. The CFDC approved businesses that would be more likely to provide ongoing employment for the interns once the intern period ended. They had 52 internships in the three years, and 44 were still working in the community last year.

  However, some respondents had less successful experiences. One respondent commented that it was hard to find youth to be interns and wondered if it related to not being able to hire interns when the intern period ended. It was noted that interns employed by not-for-profit (NFP) organizations don't usually get ongoing employment. Another respondent noted that CFDCs closer to cities or post-secondary institutions (PSIs) did not seem to have the same problem getting interns. A third respondent observed that youth internships didn't help much, as once they left to go to a PSI, they would do their internship there and stay there. It was further observed that young people want to leave and test themselves, and then, when they start a family, they want to move back. This applies to people aged 25–35 versus youth aged 18–24.

• Innovation was viewed as being successful by most respondents. Several mentioned assisting with the acquisition of process technology to improve productivity as being beneficial. Further, a few respondents commented that it is difficult to tell the difference between innovation and information and communications technology (ICT).
• ICT was viewed by respondents as being very effective. ICT projects involved getting an Internet presence (website) initially, but later focused on e-commerce and business automation (e.g., customer relationship management systems).
• Other BD projects included assistance for market diversification, branding, new trade shows, plans and studies, as noted by respondents.

Community Innovation

CI was considered successful. Several respondents noted that plans (e.g., feasibility studies, business cases, engineering and architectural studies) that were supported by EODP are being used. It was also observed that these plans and studies would not have happened if they hadn’t been 100 percent funded. Respondents noted that CI supported an NFP for foresters, which led to third-party certification, which opened up markets and increased the value of the products produced; business training for Indigenous groups; trail building for local municipalities; studies for the building of tourist attractions (e.g., Thousand Islands Boat Museum in Gananoque, Brockville Aquatarium). It was suggested that the program should be extended to things that contribute to quality of life in a community, such as a community hall.

Collaborative Projects

CP, which involved multiple CFDCs and benefited one or several communities, received a mixed reaction from respondents. Some CFDCs had been successful in implementing projects with neighbouring CFDCs, while others commented that it was difficult to put together regional projects, as there is a need to establish a consortium. This involves having to go to each CFDC, municipality, etc., to get their support, with each assessing the cost and the impact that the project will have on their local interests and how the outputs and outcomes (e.g., jobs created) will be attributed.

During EODP (2011–14), one CFDC in the consortium of CFDCs would take the lead. There was no formal approval process or dedicated funding for CP. One respondent commented that for larger projects, a longer lead time is required, as it is difficult to plan a large or strategic project and complete the implementation within a fiscal year. During the program, there were 40 projects involving the participation of 128 CFDCs (some CFDCs participated in more than one project). This was an average of 3.2 CFDCs per project, which may be considered low, given that there are 15 CFDCs in eastern Ontario that could be involved in regional projects.

5.1.4 Success Stories

The following success stories, prepared by FedDev Ontario, illustrate how EODP has assisted projects and contributed to their success.

Success Story (Winchelsea Events)—Winchelsea, located in Winchester in the United Counties of Stormont, Dundas and Glengarry (SD&G), is a multipurpose event management and catering facility. With EODP assistance through the SD&G CFDC, the company expanded its facility to accommodate an addition of 298 square metres (3,210 square feet) of serviceable space and upgrade its IT to attract a diverse range of business and special event promotional functions. Six full-time and one part-time job were created. The project was undertaken in 2013 for a total cost of $216,000, with  $50,000 repayable contribution from EODP.

Success Story (Sprucewood Brands)—Sprucewood Brands, located in Warkworth in Northumberland County, is a bakery with humble beginnings that specializes in handmade shortbread cookies. Starting out in a small kitchen in the owner's home, Sprucewood had a limited production capacity and a limited ability to keep up with demand and access new markets. With EODP assistance through the Northumberland CFDC, Sprucewood has expanded into its own 316 square metre (3,400 square foot) facility—a revitalized 1920s' building, where eight full-time and four part-time staff are now employed. This project has played a vital role in helping the business take on larger accounts, which has led to an increase in staff and plans to add more. Sprucewood is now a $500,000 per year business. The project was undertaken between 2012 and 2014 for a total cost of $79,000, with a $35,000 EODP repayable contribution.

Success Story (Fenelon Falls)—Located on the Trent–Severn Waterway in the Kawarthas, Fenelon Falls was faced with the challenge of providing improved access to its downtown commercial core for the significant boat traffic that passes through the area. With EODP assistance through the Kawartha Lakes CFDC, Fenelon Falls reconstructed a wall with mooring capability for vessels passing through the area. More than 137 metres (450 linear feet) of docking capacity was added, allowing more boats to tie up in Fenelon Falls. In addition, patrons are now within walking distance of the main street, where they can visit various restaurants, craft shops and clothing stores. In the first year alone, the economic impact is estimated to have exceeded $300,000, greatly improving the economic viability of the area. The project was undertaken in 2012 for a total cost of $187,000, with a $72,000 EODP repayable contribution.

5.2 Unintended Positive or Negative Outcomes

This subsection addresses the question "Did the EODP produce unintended positive or negative outcomes?" Respondents were asked this question during the KI interviews, and while some had no comments, others provided the following comments.

Positive

• EODP increased the presence of CFDCs in the communities. EODP also provided increased exposure for the CFDCs, which led to a higher number of loans being made and the use of their other CFDC programs—that is, increased results for their other programs. It was noted that the EODP has helped the 15 CFDCs in eastern Ontario outperform the 22 CFDCs located in southwestern Ontario, which do not have access to a similar program.
• EODP has a reputation for being straightforward and making decisions quickly.
• EODP has led to increased collaboration with other funding organizations within the community, such as Small Business Centres (Province of Ontario) and municipalities. EODP can provide support in areas where other funding is not available. This ability to provide support has led to significant opportunities for collaboration and referrals and has increased the profile of both the CFDC and EODP in the community.
• Leverage and other results were much higher than anticipated.^{Footnote 21}
• When EODP assists one business, it benefits businesses in the whole supply chain. Performance indicators do not measure this direct multiplier effect.^{Footnote 22}
• In 2011–14, with firms receiving support, there was a greater focus on economic development than on community development. Performance indicators assessing the impact of the projects led to more tangible results in the community. The performance indicators in EODP (2011–14) were improved and were more specific.

Negative

• Collaborative projects primarily involved CFDCs and it was difficult to define high-quality projects that CFDCs would support and for them to work together.
• As EODP funding for NFPs was set at 100 percent, planning studies were carried out without the funding in place to implement the projects.

6.0 Demonstration of Efficiency and Economy

This section provides the Eastern Ontario Development Program (EODP) evaluation findings for demonstration of efficiency and economy. The findings have been structured around the two key questions in Exhibit 3.1 and take into account guidance provided by the Treasury Board Secretariat on issue #5^{Footnote 23} and its three components: economy, operational efficiency and allocative efficiency. These are the two key evaluation questions for this section:

• 5.1 To what extent is the EODP delivery model efficient in producing outputs and progressing toward expected outcomes?
• 5.2 Is there a more cost-effective way of achieving the expected results, taking into consideration alternative delivery mechanisms, best practices and lesson learned?

6.1 Efficiency and Economy of the EODP Delivery Model in Producing Outputs and Outcomes

This subsection addresses the question "To what extent was the EODP delivery model efficient in producing outputs and progressing toward expected outcomes?"

6.1.1 Program Design

For program design, FedDev Ontario defines the overall scope and eligible activities and provides some guidance on areas of focus that relate to the government’s priorities. EODP (2011–14) was focused more on economic development than the prior EODP and less on community development.

The 15 Community Futures Development Corporations (CFDCs) in eastern Ontario apply to FedDev Ontario for EODP funding. Each CFDC consults with groups in their community to identify the priorities that best respond to the needs of the community and then prepares a business plan and application to obtain EODP funding. The business plan includes the CFDC’s plans and targets for each of the Business Development (BD), Community Innovation (CI) and Collaborative Projects (CP) business lines. When the application or business plan is approved by FedDev Ontario, a contribution agreement (CA) is signed with the CFDC.

EODP provides the CFDCs with the flexibility to deliver the program within its scope and eligible activities (defined by FedDev Ontario) to address their priorities, which vary by CFDC, in their geographic area. This approach is considered appropriate, as it provides each CFDC, which has knowledge of its community’s needs, with the ability to respond to local priorities.

The evaluation identified several areas of program design for consideration by program management:

• CP—This business line, which involved collaborative projects undertaken by multiple CFDCs and benefiting one or more communities, received a mixed reaction from the interview respondents. Some had been successful in implementing projects with neighbouring CFDCs, while others found it difficult to launch these types of projects. Overall, the 40 CP in EODP (2011–14) accounted for just 2 percent of the total projects and 8 percent of EODP contribution funding. EODP management had hoped for a greater number of good quality projects that were strategic and covered a larger area in the region.

  Respondents mentioned the following issues that made launching a CP project difficult: getting a group of CFDCs together to implement a joint project; complexity (e.g., no process; no application or approval; no dedicated funding, as each participating CFDC contributed some funding); varying local interests; concern about attribution of performance results; and insufficient implementation time, given that projects were required to be started and completed within the same fiscal year, which was particularly difficult for larger projects. One respondent suggested that offering the CFDCs an incentive to work together might help overcome some of the obstacles.

  Prior to EODP (2011–14), CPs were called regional projects and were delivered by the Eastern Ontario CFDC Network (EOCFDC Network).^{Footnote 24} The 2011 evaluation questioned whether the EOCFDC approach was appropriate,^{Footnote 25} and FedDev Ontario program management implemented the CP approach in EODP (2011–14) to address this concern about delivering regional projects. As discussed earlier in the evaluation, the experience with CP in EODP (2011–14) indicates that the process of delivering CP should be revisited to address some of the issues given above.

• NFP funding—Program management mentioned that 100 percent funding of not-for-profit (NFP) organizations had resulted in more planning studies being undertaken than there were funds to implement the project in planning study. The CFDC respondents did not mention a concern about the 100 percent NFP funding in EODP (2011–14), and they were able to leverage an average of $2.46 from other sources for each $1.00 of program funding (see Exhibit 5.1).
• Project size—The average project received just over $10,000 in EODP contribution funding (see Exhibit 2.2). While it is recognized that fewer, larger projects would likely involve less administration, a CFDC respondent commented that they hadn’t been successful in moving to larger projects and that smaller projects were valuable for assisting the large number of very small (micro) businesses (e.g., $300 for a training course, $1,000 for a website, $1,500 for a trade show).
• Level of CFDC contribution funding—Each of the CFDCs received an equal allocation of the EODP contribution,^{Footnote 26} with the three bilingual CFDCs being eligible to receive an additional $6,000 a year for program administration. While no interview respondents expressed concerns about the CFDCs each receiving the same amount of funding, the CFDCs have significantly different needs, demand and priorities in their areas. For example, the needs of the CFDCs along Highway 401 are significantly different from those located in more rural areas. This raises the question of whether the program funding should be allocated on a different basis that better reflects the needs and priorities of the areas covered, rather than all CFDCs receiving the same amount.

6.1.2 Demand for EODP Funding

As indicated by the number of enquiries (6,714) and the number of applications (2,683)—resulting in 2,050 projects being approved for assistance—the demand for EODP project funding was substantial. These figures indicate that, on average, 3.3 enquiries and 1.3 applications were received for each approved project.

Respondents commented that there was far more demand for assistance than funds available. While this is accurate overall, demand was stronger in some areas of the program than in others. For example, one respondent noted that skills development seems to work better with larger businesses, as small (micro) businesses didn’t seem interested. It is noted that CFDCs have the flexibility within the program guidelines to allocate their funding to areas of greatest need or interest.

From the data gathered, it is evident that the demand for EODP funding was higher than available funding, and CFDC staff has a range of qualified applicants from which to select and have the flexibility to adjust their funding allocations, as appropriate within the EODP guidelines, to areas of greatest need or priority.

6.1.3 Use of Leverage

Leverage was a requirement for all projects involving commercial enterprises, but not for NFPs:

• For BD projects involving commercial enterprises, project funding recipients were eligible to receive a non-repayable contribution not exceeding 50 percent of the total eligible project costs, up to a maximum of $100,000.
• For CI projects involving NFP organizations, project funding recipients were eligible to receive a non-repayable contribution not exceeding 100 percent of the total eligible project costs, up to a maximum of $100,000.
• For CP, project funding recipients were eligible for the amounts stipulated above.

However, as shown in Exhibit 5.1, all three business lines were able to leverage the EODP contributions. The actual leveraging considerably exceeded the target for the program, which was based on aggregating the targets provided by the CFDCs in their business plans. The EODP funding contributions of $27 million to projects resulted in leveraging additional funding of $73 million from other sources, resulting in total project expenditures of $100 million. The leveraged funding was therefore 2.71 times the EODP contributions. The leveraging for BD was higher than for CI, which in turn was higher than for CP.

Further, the 2.71 leverage achieved by EODP is comparable to the 2.65 leverage achieved by the Southern Ontario Advantage initiatives on $419.7 million of contribution expenditures through FedDev Ontario’s Southern Ontario Development Program (SODP).^{Footnote 27}

This co-investment approach, using leverage, allowed FedDev Ontario to considerably increase the impact of EODP in rural eastern Ontario communities by attracting other sources of funding.

6.1.4 Use of Authorized Funding

EODP (2011–14) was launched late, with the result that the CAs with the CFDCs were signed in late February 2012, delaying the CFDCs’ ability to fund projects. This meant that there was little time for projects to be carried out in 2011–12. FedDev Ontario recognized this problem and adjusted the cash flow for the CFDCs to $2.4 million in 2011–12 and $13.2 million in both 2012–13 and 2013–14. The delays in starting the program also led to further delays for some CFDCs, as they had to rehire staff (who had been let go earlier) to help deliver the program.

Despite the slow start, FedDev Ontario, with the assistance of the CFDCs, was successful in spending $29.9 million over the three years (see Exhibit 2.3), which was almost the entire $30 million budgeted.

In addition, projects that were supported in EODP (2011–14) had to be completed by the end of the fiscal year, and there was no carryover from one year to the next. This made it difficult to undertake larger projects or to start projects late in the fiscal year. One CFDC respondent commented that the March 31 deadline for completing projects did not give some of the funding recipients enough time to do the necessary preparatory work before embarking on their projects. Further, a respondent commented that a five-year horizon provides sufficient time to do proper planning, for partners to contribute, and to implement the solution. Another respondent stated that sometimes equipment deliveries can be delayed and miss the March 31 deadline.

6.1.5 Third-Party Delivery

FedDev Ontario made effective use of the 15 CFDCs in eastern Ontario as delivery partners for EODP. The CFDCs are a trusted delivery system and a critical component in the success of EODP. Once the CFDCs have signed the CAs, they work largely independently of FedDev Ontario program management to promote the program, evaluate requests for funding, determine the successful projects within the guidelines provided by FedDev Ontario, oversee the resulting contracts, collect performance information and pay claims.

Interview respondents, in commenting on the advantages of the CFDCs, stated that the CFDCs provide an existing structure located in the community, have staff expertise related to economic development, are aware of local issues, and have a relationship with the community. In addition, they are connected to the local businesses and NFPs and to the other players in economic development in the community. Their boards of directors are made up of approximately 12 volunteers from the community; and their staff works in the community and have for the most part been with the CFDCs for some time. The local decision making and flexible structure of the program allow it to address local challenges, within its defined scope and eligible activities.

In EODP (2011–14), the CFDCs used 7.7 percent of the contribution amount of $28.7 million for program administration activities, which is lower than the 10 percent that FedDev Ontario normally provides for third-party delivery. In the templates for the business plan, the CFDCs were limited to a maximum of $60,000 per year ($66,000 per year for the three bilingual CFDCs), for a total of $918,000 per year for the 15 CFDCs. For the three-year program, CFDC program administration activities could have cost a maximum of $2.75 million, or 9.6 percent of the total contribution that was available to the CFDCs. This indicates that the CFDCs did not budget for or charge up to the maximum amount and chose to spend the contribution funds on projects instead.

For FedDev Ontario to manage larger numbers of smaller projects in-house would require a direct interface with beneficiaries to obtain project applications, evaluate and approve the submissions, negotiate and execute project CAs, and manage the projects while they are underway. Further, considerable effort would be involved, and in some cases, FedDev Ontario’s reach and knowledge of the target organizations and individuals might be insufficient to successfully deliver the program. Moreover, effectively managing certain aspects of the program would require a local presence. For larger projects, however, where increased due diligence is required in assessing the application and monitoring the project, in-house management is required.

FedDev Ontario’s administrative cost of delivering the EODP, as shown in Exhibit 2.3, was $1.2 million for the delivery of $28.7 million in contribution funding to the CFDCs. This is $0.04 for each program dollar delivered to the CFDCs. This compares favourably with FedDev Ontario’s overall cost of $0.157^{Footnote 28} for the delivery of all its grants and contributions support programs when delivery partner costs are excluded. It also compares favourably with the delivery costs of the other regional development agencies.^{Footnote 29}

If the CFDC delivery partner cost is added to the FedDev administrative cost above, the total delivery cost would be $3.49 million^{Footnote 30} for $26.4 million in funding support provided to the project funding recipients (see Exhibit 3.1). This is $0.132 for each program dollar delivered by the CFDCs, which still compares favourably with the FedDev Ontario delivery cost of $0.157, which does not include delivery partner costs.

6.1.6 Partnerships

Partnerships expanded EODP’s impact by leveraging the EODP contributions. As noted above, each $1.00 of EODP contribution funding for projects leveraged, on average, $2.71 in funding from other sources.

The CFDC efforts are well coordinated within each of their geographic areas. They know all the other players in the community, who also know the CFDC. They work in collaboration with other funders, including other FedDev Ontario program funds, to try to determine how best to support companies. For larger projects, the organizations get together. CFDCs are often co-located with other programs in municipal buildings, and this arrangement makes for fewer stops for project proponents, facilitates synergy, reduces costs, etc.

EODP complements other funding programs, such as programs managed by the Province of Ontario. The province views EODP as being very important and complementary, as EODP works well with their funding. For example, EODP does capacity support, such as business planning and skills development, which are critical precursors to the Province of Ontario funding capital (machinery and equipment) through its own Eastern Ontario Development Fund, which is designed for much larger projects. CFDCs also worked with other funding programs, such as the National Research Council’s Industrial Research Assistance Program and the Business Development Bank of Canada. Further, EODP is complementary to the Community Futures Program (CFP). Where appropriate, CFDCs delivering CFP may reduce the loan and include a grant component from EODP. Several interview respondents noted that EODP is synergistic with CFP and resulted in the CFDCs having an increased profile in the community and better performance than CFDCs in southwestern Ontario without an EODP-type program.

6.1.7 Performance Measurement

FedDev Ontario was supported by a comprehensive performance measurement system based on the EODP Performance Measurement Strategy (2012), which had 11 outputs and outcomes and 45 performance indicators. Performance information on outputs, most immediate outcomes, and some intermediate outcomes was collected by the 15 CFDCs from the project funding recipients and then summarized and provided to FedDev Ontario. Program management at FedDev Ontario checked and entered this information into a series of spreadsheets, which facilitated consolidation and stratification by CFDC, business line and initiative and by key performance indicators.

The remaining performance indicator information would involve undertaking surveys or interviews with project funding recipients or using macroeconomic data and standardized methodology, both of which were not included in this evaluation.

6.1.8 Level of Satisfaction

CFDC respondents reported a good working relationship with the FedDev Ontario project officers and were very happy with the services received. CFDC staff consults with their project officers from time to time on grey areas, such as whether a project fits within the activity guidelines or how to classify it for performance tracking. FedDev Ontario project officers respond very quickly.

CFDCs reported that they provide an appropriate level of support to project applicants; help fill out application forms, etc.; and work with the funding recipients. As project funding recipients were not consulted in this calibrated evaluation, no information was available on their experiences or level of satisfaction. However, FedDev Ontario program management stated that they have received very few complaints, and no major problems had arisen.

6.1.9 Administrative Processes

• Promotion—EODP is well known in communities by interested parties. To reach potential applicants for funding, CFDCs reported using a variety of approaches: advertising in print media, using website and social media, word-of-mouth, attending meetings, promoting success stories, outreach through other economic organizations in the community, and so on.
• Project identification and selection—Project identification and selection processes, templates, etc. varied across the CFDCs. For example, CFDCs used different types of intake (continuous, monthly, etc.), different approaches, and different criteria in evaluating and selecting projects. Each CFDC has invested in developing these processes. An "Other" respondent stated that there needs to be greater transparency in the use of EODP funds by the CFDCs, such as the selection process.
• Delivery cost—Almost all CFDCs expressed concern that the maximum $60,000 per year for administration was tight or unreasonable. They provided illustrations of why the incremental costs were at least this amount, given the salary and benefits of a full-time EODP coordinator and costs for marketing, audit, office, etc. Several mentioned that the program was being subsidized by the CFDC. One exception was a CFDC respondent who considered the $60,000 to be very reasonable for a part-time EODP coordinator and for advertising. Further, as noted in Section 6.1.5 above, the financial information indicates that the CFDCs did not budget for or charge up to the maximum amount allowed under EODP and chose to spend the contribution funds on projects instead.
• Amount of time—Most CFDCs noted that the delay in receiving funding from FedDev Ontario in 2011–12 was a major problem. However, when funding was available, the CFDCs were able to process applications quickly, taking two to six weeks to provide a proponent with a response.

Administrative processes and templates have been developed by each CFDC independently. While a few CFDCs mentioned talking to other CFDCs about processes, there may be potential benefits in making available best practices and lessons learned in key administrative areas, with the objective of improving efficiency. This might be a potential role for the EOCFDC Network.

6.2 Cost-Effectiveness in Achieving Expected Results

This subsection addresses the question "Is there a more cost-effective way of achieving the expected results, taking into consideration alternative delivery mechanisms, best practices and lessons learned?"

There was a consensus among interview respondents that FedDev Ontario’s use of CFDCs to deliver EODP was the most cost-effective approach of achieving program results. One respondent noted that there is no better, more cost-effective group of organizations that are lean, well established and fully connected in the community. Another commented that there is no better delivery method at the local level that could ensure the quality of applications and with a board of directors from the local area with knowledge of the local business and NFP community.

Further, the costs incurred by FedDev Ontario in using the CFDCs to deliver EODP were largely incremental to CFDC operations, and as shown above, the delivery costs for EODP were found to be very reasonable (see Section 6.1.5 above). Moreover, EODP was synergistic with other CFDC operations, as it improved a CFDC’s overall performance and increased CFDC profile in the community.

7.0 Conclusions and Recommendations

This evaluation was undertaken to determine the extent to which the Eastern Ontario Development Program (EODP) continues to be relevant, is on track to achieve its expected program outcomes, and has demonstrated efficiency and economy. The evaluation involved a calibrated methodology that took into account timing of previous evaluations and changes to the EODP. Conclusions and recommendations for the evaluation are provided below.

7.1 Conclusions

The following conclusions are based on the evidence collected in this evaluation.

Program Relevance

Past evaluations and the government’s renewal of the program for the 2014–19 period confirm EODP’s relevance.

Interview respondents were generally satisfied with EODP (2011–14). They supported continuing the program, as it stimulated investment and played a key role in the community. Further, respondents felt that although EODP resources were small in comparison to the size of the rural eastern Ontario economy, the program was extremely effective and had an impact by helping address major challenges and encouraging business development and community development.

EODP was found to complement other federal and provincial government funding programs where they were available to the project funding recipients. The existence of other programs was not viewed as a concern, as other programs had a different emphasis and the demand for funding in eastern Ontario was much greater than the limited funding available from EODP.

EODP (2011–14) was found to be consistent and fully aligned with government priorities, including FedDev Ontario’s Program Alignment Architecture and strategic outcome; and federal government priorities and strategies, including Budget 2010 (March) and Budget 2011 (June). Further, EODP was fully aligned with federal roles and responsibilities.

Program Performance—Achievement of Program Outcomes

Overall, EODP (2011–14) was generally successful in achieving results and exceeding almost all targets established for the program. The business lines Business Development (BD) and Community Innovation (CI) were found by respondents to have been successful, with some caveats (section 3.3). Collaborative Projects (CP), which involved investments from two or more Community Futures Development Corporations (CFDCs) and benefited one or more communities, had a mixed result, with some respondents feeling it had been successful and others feeling it had significant challenges.

Performance information indicates that the program was successful in building partnerships, supporting businesses and communities, creating and maintaining jobs, and contributing to diversification and competitiveness.

Interview respondents viewed the CFDCs as being effective in contributing to the rural economy and community by supporting small business development; leveraging funding from other programs; and working with partners to promote economic stability, growth and diversification. Moreover, several respondents noted that given EODP’s limited budget, the program had been extremely effective in achieving results. However, while these results can be seen at the community level, it is difficult to assess the program’s impact on eastern Ontario given the size of the eastern Ontario economy. In addition, interview respondents noted a number of unintended positive outcomes, as well as a few challenges resulting from the program.

Program Performance—Efficiency and Economy

The EODP (2011–14) delivery model was found to have been efficient and economical in producing outputs and progressing toward expected outcomes. The delivery model was considered appropriate, as it provided each CFDC, which has knowledge of the community’s needs, with the ability to respond to local priorities.

The demand for EODP funding was much higher than available funding, and program staff had a range of qualified applicants from which to select. Partnerships and leverage, including third-party delivery (i.e., the CFDCs), were found to be instrumental in expanding FedDev Ontario’s impact. The requirement that supported projects had to be completed by the end of the fiscal year and could not be carried over from one year to the next created problems and inefficiencies for the CFDCs. Further, administrative processes and templates have been developed by each CFDC independently, and there was considerable variability among the CFDCs.

Third-party delivery involved trusted partners (e.g., CFDCs) and was found to be both efficient and effective. The cost of delivering an EODP dollar was found to compare favourably with the overall delivery cost of grants and contributions programs by FedDev Ontario and by other regional development agencies. CFDC respondents reported a good working relationship with the FedDev Ontario project officers and were very happy with the services received. The delays in starting the program caused difficulties for the CFDCs; however, they successfully delivered the program in the time remaining in the three-year period.

There was a consensus among interview respondents that FedDev Ontario’s use of CFDCs to deliver EODP was the most cost-effective approach to achieving program results. It was further observed that there is no better, more cost-effective group of organizations that are lean, well established and fully connected in the community.

7.2 Recommendations

The following recommendations, based on the evaluation findings, are made for consideration by FedDev Ontario management.

Program Design

1. It is recommended that FedDev Ontario program management undertake the following:

1. Investigate whether there is a better approach to planning and managing Collaborative Projects (i.e., regional projects), which involve funding from two or more CFDCs and benefit one or more communities, to address the concerns identified in this evaluation.
2. Examine approaches to reduce the number of not-for-profit (NFP) planning studies for projects that do not have available funds to implement them.
3. Assess the relative value and balance of small projects versus large projects, given the decision to encourage larger strategic projects in 2013–14 and the fact that most businesses in eastern Ontario are small (micro) businesses with one or very few employees and do not require larger funding amounts.
4. Review the funding approach where each CFDC receives equal contribution funding to determine whether funding should be allocated on a different basis that reflects the needs and priorities of the geographic areas covered.

Third-Party Delivery

2. It is recommended that FedDev Ontario program management undertake the following:

1. Revise the funding approach in which supported projects must be completed by the end of the fiscal year that they start in and allow carryover to accommodate late-starting and multi-year projects.
2. Encourage the CFDCs to benefit from the various approaches (e.g., processes, templates, software) developed and used by the CFDCs across eastern Ontario for EODP management and administration (promotion, application processing, project evaluation and selection, etc.) by adopting the best practices and lessons learned, with the objective of improving efficiency. The Eastern Ontario CFDC Network could potentially lead this initiative.

Economic Research

3. It is recommended that FedDev Ontario management undertake a study of the socioeconomic situation in eastern Ontario to help determine the continuing need for a program similar to EODP as input to future program renewal. This study should be carried out after the 2016 Census information becomes available. This study could also include determining if there is merit in extending the program to areas with similar economic needs in southwestern Ontario.

Intermediate and Ultimate Outcomes

4. It is recommended that FedDev Ontario management include the following methodologies in future evaluations of EODP:

1. Assess the longer term benefits to the project funding recipients. Also, investigate the role that EODP funding has in launching the supported projects; and determine the project funding recipients’ level of satisfaction and feedback in relation to the delivery of the program. This would likely include a survey of project funding recipients, interviews with CFDCs and project funding recipients, and case studies of supported projects.
2. Assess the program’s impact on the ultimate outcomes, which would involve undertaking special analyses using Statistics Canada data, subject to data availability.

Selected References

1. EODP Draft Evaluation Methodology Report, FedDev Ontario, 2015.
2. Evaluation of the Community Futures Program, FedDev Ontario, 2014.
3. EODP Performance Measurement Strategy (PMS), FedDev Ontario, 2012.
4. Evaluation of the Eastern Ontario Development Program, FedDev Ontario, 2011.
5. Interim Evaluation of FedDev Programs, Westbay Research Inc., October 2012.
6. Labour Market Bulletin—Ontario: 2012 (Annual Edition), Statistics Canada.
7. Labour Market Bulletin—Ontario: 2014 (Annual Edition), Statistics Canada.
8. Eastern Ontario’s Economic Development Strategy, Miller Dickinson Blais, February 2014.
9. 2014 Economic Scan of Eastern Ontario, McSweeney & Associates, October 2014.
10. Federal Economic Development Agency for Southern Ontario, 2013–14 Report on Plans and Priorities, FedDev Ontario.
11. Selected Documents on the Evaluation Function, Treasury Board Secretariat, 2009.
12. Federal Government Budgets, 2010 to 2013.
13. Analysis of the Federal–Provincial Economic Development Program Landscape, Intergovernmental and Stakeholder Relations, FedDev Ontario, June and September, 2012.
14. Ontario Economic Update 2016, Ontario Chamber of Commerce.

Appendix A: 2011 EODP Evaluation: Recommendations, Management Responses and Actions Taken

Recommendation	Management Response Reported to Audit and Evaluation Branch	Responsible Official	Revised Completion Date	Action Taken (as of December 2015)
Recommendation #1: FedDev Ontario should examine whether it is necessary to have four agreements for the administration of the EODP funds and determine if all of the funds could be administered through one agreement.	Agreed. FedDev Ontario will streamline the contribution agreement for the delivery of the next generation of EODP so that a single agreement per Community Futures Development Corporation (CFDC) is used to fund all program components. This approach is consistent with The Government of Canada Action Plan to Reform the Administration of Grant and Contribution Programs, with a focus on simplification of funding programs and agreements.	Director, Community Economic Development	Fall 2011	Addressed. The four agreements per CFDC were reduced to one agreement per CFDC for EODP (2011–14).
Recommendation #2: FedDev Ontario should review its current Performance Measurement Strategy (PMS) and refine its data collection to ensure it is appropriate for measuring the outcomes of the program. Including CFDCs and academics in the review and identification of indicators would be an effective way of ensuring the appropriateness of the indicators for measuring success.	Agreed. FedDev Ontario will redesign the EODP PMS. The number of performance indicators will be reduced to simplify reporting and to ensure data collection is relevant and appropriate for measuring the outcomes of the program. These efforts will be informed by the work toward consolidation and integration of performance measurement strategies for programs across the Agency.	Director General, Strategic Policy Director General, Innovation and Economic Development Director General, Audit and Evaluation	Fall 2011	Addressed. The EODP (2011–14) PMS was redesigned. In addition to the program no longer including access to capital, the number of performance indicators was reduced from 99 to 45 to simplify reporting while ensuring that data collection was relevant and appropriate for measuring the program outcomes.
Recommendation #3: FedDev Ontario and CFDCs should ensure that a standardized, up-to-date list of recipients is maintained. This list should include complete information on the recipients (e.g., name, email address) as well as specific project identifiers (e.g., project name, project type). This information will be required should FedDev Ontario wish to survey its recipient population for future evaluations. Alternatively, FedDev Ontario could implement a post-project feedback form, which could be administered to recipients following the completion of the project. This form could collect information on the impacts of the project. The results could be entered into a database and be used for both ongoing management and evaluation purposes.	Agreed. FedDev Ontario will explore project and client management technology that will assist with ongoing monitoring and tracking data. A software/database program option will be investigated that could allow real-time access to project data and recipient information.	Director, Community Economic Development	Fall 2011	Addressed. As part of the EODP (2011–14) PMS redesign, the performance measurements tools were updated to incorporate recipients’ contact information along with the project impacts. The data was collected from the CFDCs’ quarterly reports and incorporated into a master database. FedDev is currently working with other regional development agencies (RDAs) on a common Enterprise Grants and Contributions Solution, with an anticipated release date in 2018.
The evaluation raises questions about whether the current model for the delivery of regional projects is appropriate. Recommendation #4: Since this issue has been identified in previous evaluations, FedDev Ontario should examine alternative delivery options for regional projects to determine whether they could be delivered more efficiently.	Agreed. FedDev Ontario will review the experience related to the delivery of the EODP regional projects, with the objective of determining a more efficient approach for the delivery of this component of the program. This will include the elimination of a separate and unique process for regional projects.	Director, Community Economic Development	Fall 2011	Addressed. Regional projects were not included in EODP (2011–14); however, a new category of CFDC projects, Collaborative Projects, was introduced. The intention was that CFDCs could use their EODP funding to work together on joint projects.

Appendix B: Eligible EODP (2011–14) Recipient Activities

The following descriptions of eligible EODP (2011–14) project funding recipient activities were extracted from the signed contribution agreements (CAs) between FedDev Ontario and each Community Futures Development Corporation (CFDC). The three business lines, which are described below, are as follows:

• Business Development (BD)
• Community Innovation (CI)
• Collaborative Projects (CP)

Eligible project funding recipients included the following:

1. Non-for-profit organizations, including municipalities and municipal organizations, corporations, and community development organizations;
2. Commercial enterprises, including individuals, corporations, partnerships, cooperatives and trusts; and
3. Indigenous organizations.

B.1.0 Business Development

The BD business line supports projects that will lead to the growth of new and existing businesses within rural eastern Ontario communities. This business line includes initiatives in key areas, such as skills development, youth attraction, innovation, and information and communications technology (ICT) adoption.

Eligible Activities

• Business planning and business development initiatives, such as business development plans, business expansion plans, marketing strategies, export plans, feasibility studies, and strategic plans;
• Research and development (R&D) initiatives and research analysis;
• Internship and job placement initiatives, such as youth internships and non-youth internships (see below);
• Trade show attendance in support of market expansion;
• Skills development and training, such as incremental on-the-job training, in-house or outsourced classroom training, business management skills development, workshops and seminars;
• Business and trade opportunity development;
• Marketing projects;
• Economic, tourism and business development activities;
• Innovation initiatives, including commercialization of new technologies, innovation and technology R&D, technology or ICT, and knowledge-based economic studies and plans;
• Technological enhancement initiatives, such as advanced innovative web enhancements, e-commerce, e-learning platforms, intranet and extranet portals, geographic information system applications, customized software development, and installations to improve efficiencies and productivity;
• ICT infrastructure projects, including telecommunications improvements, broadband and Internet access;
• Applications in ICT, which may include development of networks, ICT education and awareness, and facilitation or hosting of conferences and workshops; and
• Other incremental activities designed to stimulate business opportunities in the knowledge-based economy.

Internship and Job Placement Initiatives

• Youth interns employed through agreements with eligible project recipients will normally be graduates with a degree, diploma or Ontario Ministry of Education certificate from a secondary or post-secondary institution and be under 30 years of age. Intern projects and positions will not normally exceed 12 months in duration. All exceptions require prior written approval from FedDev Ontario.
• Non-youth interns employed through agreements with eligible project recipients will normally be graduates from a secondary or post-secondary institution and be over 30 years of age. Intern projects and positions will not normally exceed 12 months in duration. All exceptions require prior written approval from FedDev Ontario.

Ineligible Activities

• Hiring of interns for administrative or operational activities considered part of regular operations;
• Hiring of interns that displace existing employees;
• Repayable financial assistance; and
• Internships that do not comply with labour union agreements.

B.2.0 Community Innovation

The CI business line supports community-led economic development that enhances and diversifies local economies. Working in partnership with community stakeholders, CI builds the economic and innovative capacity of rural eastern Ontario communities, and develops opportunities for sustainable economic growth and employment.

Eligible Activities

Eligible activities are CI project activities that

• Contribute to achieving EODP objectives by supporting community-based initiatives that stimulate business and community development opportunities, promote socioeconomic development leading to a competitive and diversified regional economy, and contribute to the successful development of business and job opportunities and sustainable, self-reliant communities;
• Demonstrate broad support from the respective community; and
• Offer sustainable and measurable benefits to the local or regional economy.

Examples of Eligible Activities

• Community-based economic development projects;
• Research projects;
• Marketing and promotional activities;
• Tourism events;
• Export and trade initiatives;
• Facilitation or hosting of seminars or workshops;
• Small-scale construction, projects or physical works; and
• Other activities in support of economic development.

B.3.0 Collaborative Projects

EODP projects that involve and benefit one or more communities are encouraged. The CP business line is expected to result in a greater number of strategic economic development initiatives and improved partnerships that benefit eastern Ontario communities, including the strengthening of linkages between urban and rural areas. The objective of these projects is to support initiatives that will stimulate BD and CI opportunities by promoting socioeconomic development leading to a competitive and diversified regional economy within eastern Ontario. These projects will involve investments from two or more CFDCs.

Eligible Activities

• CP initiatives should promote broad-based collaborative economic development leading to a competitive and diversified economy while contributing to the successful development of business and job opportunities and prosperous communities. The initiatives should demonstrate broad support from the region or various partners; and offer sustainable, measurable benefits to the regional economy. Collaborative projects will have EODP investment support and contributions from two or more CFDCs.
• Special consideration should be given to projects that focus on initiatives that stimulate business development and job creation; support community innovation; attract and retain youth; increase opportunities for skills development; and enhance innovation and ICT.
• Eligible activities include those specified in Section 1 of this Schedule (Business Development) and Section 2 of this Schedule (Community Innovation).

Appendix C: Program Management

The following describes the roles and responsibilities of FedDev Ontario and the Community Futures Development Corporations (CFDCs) in the delivery of EOFP (2011–14).

C.1.0 FedDev Ontario

FedDev Ontario was responsible for the overall management of EODP. FedDev Ontario EODP program officers worked with the CFDCs to support them in effectively administering the locally based initiatives. Specific responsibilities included the following:

• Establishing overall objectives for EODP;
• Reviewing and approving CFDC applications for EODP funding on the basis of business plans reflecting local or regional needs, priorities and opportunities;
• Monitoring the outcomes of program activities;
• Overseeing the delivery of EODP by CFDCs according to a risk assessment that determines the extent of monitoring and review required by FedDev Ontario;
• Ensuring program reporting is in line with the whole-of-government framework; and
• Reviewing and monitoring the overall progress of EODP.

C.2.0 Community Futures Development Corporations

CFDCs in eastern Ontario applied for and subsequently received EODP funding through their own contribution agreement (CA) with FedDev Ontario. In addition to the development of a business plan and application, specific roles and responsibilities were set out in the terms and conditions of the CA. Specific responsibilities included the following:

• Collaborating with potential partners to establish short- and long-term priorities for EODP funding, as well as achievable targets or outcomes;
• Preparing a business plan and application to obtain EODP funding;
• Collaborating with potential partners to develop funding proposals;
• Managing the contribution provided under the CA with FedDev Ontario in support of eligible projects;
• Assessing and selecting eligible project recipients in an open and fair manner;
• Making decisions regarding project approvals and entering into project CAs with eligible recipients when distributing EODP funds;
• Providing financial assistance from the EODP funds to the project recipients, as appropriate;
• Implementing project activities either alone or in partnership with other organizations and conducting due-diligence assessments as required;
• Preparing and managing the project CAs;
• Establishing administrative policies and procedures to be used for the administration of the project CAs;
• Submitting substantiated claims for payment, progress reports, and a final project summary report to FedDev Ontario; and
• Providing administrative capacity and quality control in support of EODP delivery.

Appendix D: EODP Logic Model and Performance Indicators

The logic model developed for EODP is given in Exhibit D.1. The logic model shows the activities that were undertaken in the program, the outputs resulting from those activities, and the immediate, intermediate and ultimate outcomes each of the initiatives was intended to achieve.

In total, there were 11 outputs and outcomes and 45 performance indicators for the EODP.

Exhibit D.1: EODP Logic Model (2012)

Source: FedDev Ontario, Performance Measurement Strategy for the Eastern Ontario Development Program (January 2012), p. 8.

Exhibit D.2: Performance Measurement Matrix

Outputs

Logic Model Component	Indicators*	Source/Frequency
Policies, plans, reports and program tools, and communication resources	# of program policies developed # of program tools and resources developed	FedDev files (Delivery unit) / ongoing
Advice, information and support	# of info sessions, workshops or training sessions held with primary recipients, communities and other stakeholders # of participants attending info sessions, workshops or training sessions # of monitoring interactions between FedDev and primary recipients	FedDev files (Delivery Unit) / survey ongoing
Contribution agreements and funding key partners	# and $ value of agreements with Community Futures Development Corporation (CFDCs) # and $ value of agreements with other eligible recipients	FedDev files (Payment and Monitoring / GCRS) / ongoing

Immediate outcomes

Logic Model Component	Indicators*	Source/Frequency
Improved business development	# and $ value of business development projects # and $ value of marketing or export plans developed # and $ value of strategic plans developed # and $ value of feasibility studies completed # and $ value of business plans developed Identification of new markets and new opportunities $ value leveraged from other partners	FedDev files (Payment and Monitoring / GCRS/CEDD) / Recipient project reports / surveys ongoing
Improved community innovation	# of new staff and volunteers in community organizations dedicated to community economic development # of community or organizational plans developed # and $ value of local community economic development projects Extent to which EODP youth internship projects contribute to community economic development # and $ value leveraged from other partners	FedDev files (Payment and Monitoring / GCRS/CEDD) / Recipient project reports ongoing

Intermediate outcomes

Logic Model Component	Indicators*	Source/Frequency
Employment is created or retained	# of jobs created and retained as a result of EODP funding # and % of interns with ongoing employment in rural eastern Ontario # and % interns (youth or non-youth) interested in starting own business or working in small business environment # of interns subsequently hired by local business Extent to which EODP projects have led communities to identify or secure future labour needs	Recipient project reports / survey of clients / key informant interviews ongoing / annual review
Businesses are created, retained, expanded, or attracted	# of businesses created (incl those attracted), maintained or expanded as a result of EODP funding # of projects contributing to the development of new or external markets $ and % increased revenues due to EODP funding $ and % new or start-up revenue due to EODP funding $ and % of increased profits due to EODP funding $ and % of increased export revenues due to EODP funding Survival rate of business	Recipient project reports/ survey of CFDC loan recipients/ key informant interviews ongoing/annual review
Economic sectors are created or strengthened	# of new economic sectors active in community as a result of EODP funding # and $ value of projects developing and deploying information and communications technology ICT (including knowledge-building projects) # and $ value of projects contributing to innovation (e.g., new products or processes) $ and % profit growth in existing economic sectors due to EODP funding Extent to which EODP projects have led to improved learning opportunities with respect to economic diversification	FedDev files (CEDD)/ Recipient project reports/ surveys ongoing/annual review

Ultimate outcomes

Logic Model Component	Indicators*	Source/Frequency
Diversification and competitiveness of rural eastern Ontario economy	# of regional, sub-regional and local partnership initiatives in place to respond to barriers to rural development Increased # of employed persons working in various industry sectors GDP growth in various sectors of the economy	Key informant interviews / Census, Statistics Canada Labour Force Survey / 2–5 years
Economic stability/growth and job creation	Employment levels relative to other rural or urban communities Increasing trends in earnings data # of businesses and organizations increasing employment, payrolls and hours	Census/ Statistics Canada Labour Force Survey / Survey of Employment, Payrolls and Hours / 2–5 years
Sustainable, self-reliant communities	Steady or growing municipal tax base Increasing per capita income Increasing educational attainment of residents	Census/ Statistics Canada Labour Force Survey/ 2–5 years

* Indicators measuring the impact of EODP programming on the socioeconomic development of official language minority communities will be incorporated into CFDC project reports to be consistent with relevant policies and regulations of the Official Languages Act.

Note: CEDD, Community Economic Directorate; GCRS, Grants and Contributions Reporting System.

Source: FedDev Ontario, Performance Measurement Strategy for the Eastern Ontario Development Program (January 2012), p. 13.