Evaluation of the Community Futures Program

Report Prepared for the Federal Economic Development Agency for Southern Ontario, April 28, 2014

Prepared By:
Ference Weicker & Company Ltd.
550-475 West Georgia Street
Vancouver, BC
V6B 4M9
604-688-2424

Table of Contents

List of Tables

  1. Communities Selected for Case Studies
  2. CFP Expenditures
  3. Number and Percentage of Loan and Non-Loan Clients Reporting Significant Assistance other than Financing from Another Program (Types Of Assistance)
  4. Summary of New and Active Loans by Year
  5. Summary of New Loans by CFDC Region, 2009–102012–13
  6. Loan and Non-Loan Client Perceived Impacts Generated by the CFDC Assistance
  7. Profile of CED Initiatives Supported by CFDCs, 2009–10 to 2012–13
  8. Funding for CFDCs from the Community Adjustment Fund and the Eastern Ontario Development Program, 2009–10 to 2012–13
  9. Calculation of Economic Benefits Attributable to the CFDC Loans in the Fifth Year After the Loan and in the Aggregated Five Year Period
  10. Number of Jobs Created and Maintained as Reported Annually by the CFDCs
  11. Cost per Outputs/Outcomes Across RDAs for 2010–11 and 2011–12
  12. FedDev Ontario CFP Cost per Output/Outcome 2009–10 to 2012–13
  13. CFDC Operational Funding Analysis, 2012–13
  14. Percentage of the Investment Funds in Active Loans
  15. Investment Portfolio Growth, 2009–10 to 2012–13
  16. CFDC Loan Loss Rates, 2004–05 to 2011–12
  17. Overview of FedDev Ontario
  18. Evaluation Issues and Questions
  19. Response Rate by Type of Client
  20. Involvement of Community Partners with the CFDCs
  21. Involvement of FedDev Ontario Representatives in the CFP
  22. Involvement of Other Stakeholders with the CFP
  23. Communities Selected for Case Studies
  24. Number of Focus Group Participants, Surveys and Interviews Conducted in Each Case Study Region

List of Figures

  1. Types of Services Received by CFDC Clients Surveyed
  2. Community Futures Program Logic Model

List of Acronyms

ABDC Aboriginal Business Development Canada
ACOA Atlantic Canada Opportunities Agency
BDC Business Development Bank of Canada
CAF Community Adjustment Fund
CED Community Economic Development
CED-Q Economic Development Agency of Canada for the Regions of Quebec
CFDC Community Futures Development Corporation
CFP Community Futures Program
CMA Census Metropolitan Area
CYBF Canada Youth Business Foundation
EDO Economic Development Officer
EOCFDC Eastern Ontario Community Futures Development Corporations Network Inc.
EODF Eastern Ontario Development Fund
EODP Eastern Ontario Development Program
ESDC Employment and Social Development Canada
FedDev Ontario Federal Economic Development Agency for Southern Ontario
FedNor Federal Economic Development Initiative for Northern Ontario
IC Industry Canada
KPI Key Performance Indicator
LFS Labour Force Survey
NAICS North American Industry Classification System
OACFDC Ontario Association of Community Futures Development Corporations
OMAFRA Ontario Ministry of Agriculture and Food and Ministry of Rural Affairs
PAA Program Alignment Architecture
PBFM Performance-Based Funding Model
RDA Regional Development Agency
RED Rural Economic Development Program
RIC Regional Innovation Centre
ROMA Rural Ontario Municipal Association
SBEC Small Business Enterprise Centre
SME Small- and medium-sized enterprise
SODF Southwestern Ontario Development Fund
SODP Southern Ontario Development Program
SOFII Southern Ontario Fund for Innovation and Investment
TBS Treasury Board of Canada Secretariat
WD Western Economic Diversification Canada
WOCFDCA Western Ontario Community Futures Development Corporations Association

top of page

Executive summary

Overview of the evaluation

Established as a national program in 1985, the Community Futures Program (CFP) has been administered in southern Ontario by the Federal Economic Development Agency for Southern Ontario (FedDev Ontario) since 2009. The objective of the CFP is to promote economic stability, growth and job creation leading to more diversified and competitive rural economies, and economically sustainable communities. In southern Ontario, the program is delivered through a network of 37 Community Futures Development Corporations (CFDCs). From 2009–10 to 2012–13, program expenditures, including grants and contributions and operations and maintenance, totalled $50.31 million. The support provided by FedDev Ontario enables the CFDCs to provide financing and other services to small businesses as well as to work with local partners on community strategic planning and economic development initiatives.

The objective of this evaluation is to report on the relevance and performance of the CFP in southern Ontario, in accordance with the 2009 Treasury Board of Canada Secretariat (TBS) Policy on Evaluation, Directive and Standards. The scope of the evaluation includes CFP activities funded from 2009–10 to 2012–13. The evaluation approach is based on the national Performance Measurement Strategy (PMS) approved for the CFP.

The evaluation employed multiple lines of evidence including surveys of 512 CFDC clients (309 clients who received CFDC loans and 203 non-loan clients), 79 CFDC representatives (staff and board members of CFDCs) and 73 community partners, interviews with 12 FedDev Ontario representatives and 16 other stakeholders, a document and literature review, analysis of administrative data, and four case studies focused on specific CFDCs. The case studies included document reviews, obtaining additional input from 33 representatives through interviews and focus groups, and an analysis of 110 interviews or surveys already conducted with representatives in the communities.

Major findings

The major findings arising from the evaluation of the Community Futures Program in southern Ontario are as follows:

Relevance

Continued Need for the Program

1. There is a strong need for all four lines of business of the CFP in southern Ontario, given the significant economic challenges faced by smaller communities, the need for support targeted specifically at small- and medium-sized enterprises (SMEs), and the important role that CFDCs play in supporting community economic development. Smaller communities face significant economic challenges associated with a low and often declining population base, more restricted access to workers, particularly skilled workers, and a heavy reliance on natural resource sectors which makes the economy more vulnerable to economic swings.

There is a continued need to provide business financing and business information and services for SMEs. Support targeted specifically at SMEs is needed because smaller centres are much more reliant on SMEs for business activity and employment, the entrepreneurial base is aging and will need to be replaced, SMEs have much more restricted access to loan financing, and rural areas tend to have more limited access to business services.

There is also a need to enhance the capacity of local and regional governments, catalyze partners, and leverage funding from sources other than the CFP to support community strategic planning and community economic development (CED) projects. While providing support directly to small businesses is an important strategy for promoting community economic development, many key issues are best addressed through broad systemic initiatives in partnership with local governments. CFDCs play an important role as a catalyst for such initiatives, building partnerships and bringing the capacity, expertise, support, and business perspective needed to move community plans and CED initiatives forward, particularly in regions where local organizations have limited capacity for economic development.

Relationship to Other Programming

2. The activities of the CFDCs complement rather than duplicate or overlap other resources that are available. Numerous other government programs are available in southern Ontario which share objectives similar to those of the CFP, including programs that provide small business financing and other business services, and support community strategic planning and CED initiatives. The greatest potential for overlap lies in the provision of basic business information and services as well as larger loans. The actual level of duplication and overlap is low because of the strong demand for services and the significant coordination, collaboration, leveraging of resources and differentiation that exists between the programs. The flexible design of the CFP enables CFDC activities to be tailored to the specific needs and gaps in services within the local communities.

Alignment with Federal Priorities, Roles & Responsibilities

3. The objectives of the CFP are aligned with the strategic outcomes of FedDev Ontario as well as with federal government priorities, roles and responsibilities. CFP objectives align well with FedDev Ontario's strategic outcomes, particularly with respect to sustained economic development, growth of communities and "a competitive southern Ontario economy." The objectives also align with recent Speeches from the Throne that echo the Government of Canada's commitment to job creation, support to small businesses, and the economic well-being of communities. The programming is also consistent with the commitment of the federal government, under the Constitution Act (1982), to promote "economic development to reduce disparity in opportunities."

Performance – Effectiveness

Financing and Business Development Services

4. From 2009–10 to 2012–13, the CFDCs provided a range of services to approximately 2,300 loan clients including 2,779 loans valued at over $165 million. Clients who were surveyed reported investing an average of $1.50 in funding from other sources for every dollar loaned by the CFDC. A greater number and value of loans were issued per CFDC by CFDCs in the East compared to those in the West, in part, due to differences in the availability of other services. In addition to loans, 54% of loan clients surveyed also received one-on-one business counselling, 42% participated in a training program, 41% received business plan assistance, and 35% received business information. These services helped clients to further develop their skills (reported by 34% of clients), access other programs and services (31%), improve business practices (29%), or start their own business (24%). The CFP had a significant impact in increasing access to capital; a majority of loan recipients were turned down for funding from other sources or believe it is unlikely that they could have accessed funding from other sources.

5. CFDC loan clients grew faster, had higher rates of survival, and generated an estimated additional $516 million in revenues, 3,865 jobs, and almost $130 million in wages over a five year period compared to similar businesses that did not receive loans. The Business Number analysis using Statistics Canada data indicates that SMEs which received loans from CFDCs in southern Ontario grow significantly faster (average revenue growth of 14.9% vs. 6.9%) and have higher survival rates (88% vs. 66%) after five yearsFootnote 1 than do similar businesses that did not receive CFDC loans. Using the results of the client survey and the Business Number analysis, it is estimated that, in comparison to the control group, CFDC loan clients generated an additional $516 million in revenues, 3,865 jobs, and almost $130 million in wages over a five year period.

6. CFDC clients which only received business information and services improved their business skills and practices, started and further developed their business, and expanded their access to other programs and services. However, the impact of the services is unquantifiable due to a lack of adequate data. Of the non-loan clients who were surveyed, 29% received one-on-one business counselling, 29% received business plan assistance, 27% participated in training, and 27% received business information. These services helped clients to further develop their business skills (identified by 39% of clients), start their own business (33%), improve business practices (30%), and increase access to other programs and services (25%). Sixty-three percent of the non-loan clients surveyed are operating a business and, on average, the revenues of businesses have increased by an average of 15% per year since the services were received. Over two-thirds of non-loan clients rated the services as important or very important in the further development of the business. In addition, case studies demonstrated CFDC efforts to enhance knowledge, skills, and awareness of entrepreneurship among youth. However, there is no reliable basis on which to extrapolate the results to the total population of non-loan clients.

Strategic Planning, CED Projects and Partnerships

7. The CFP plays an important role in community economic development and strategic planning, particularly in terms of in-kind support from CFDC staff (e.g., in leveraging funding from other government programs and partners, catalyzing partners, and enhancing the capacity of local and regional governments). Almost all CFDCs (92%, 93% in the East and 91% in the West) have led CED projects over the past four years while 59% of CFDCs (60% in the East and 59% in the West) have taken the lead in the development of one or more community strategic plans. The CFDCs reported involvement in 2,317 CED initiatives, 30% of which they led. The CFDCs contributed $8.7 million to CED initiatives (mostly from sources other than their CFP base budget, particularly the Eastern Ontario Development Program (EODP) and the Community Adjustment Fund [CAF]), which was leveraged with $45.5 million in funding from other partners (average of three partners per project). CFDCs in the eastern compared to the western region have been more active in CED projects as a result of receiving more CED funding from other programs (e.g., EODP funding to eastern CFDCs totaled four times the level of CAF funding to western CFDCs per CFDC over the four years of the evaluation). Key informants and case studies demonstrated ways CFDCs have been successful in strengthening community capacity for socio-economic development by increasing access to capital, strengthening the entrepreneurial base, and building the economic development capacity of local organizations.

Ultimate Outcomes

8. Most key informants view the CFDCs as effective in supporting small business development, leveraging funding from other programs, and working with partners to promote economic stability, growth, and diversification. According to all groups of key informants, the CFP has been particularly successful in facilitating small business development ($516 million in new revenues), the creation or maintenance of jobs (3,865 new jobs), economic stability, and diversification. The case studies demonstrated how CFDCs have worked with various partner organizations in delivering services, leveraging provincial and federal funding, developing strategies and implementing CED initiatives to support economic growth (e.g., business attraction), stability and diversification (e.g., helping businesses to reduce costs and supporting new job sectors). Some key informants noted that CED and community strategic planning efforts have been more successful than business financing and services in supporting diversification, innovation and competitiveness.

Contributing and Constraining Factors

9. The major factors which contribute to the effectiveness of the CFP include the flexible, locally based delivery model, the experienced and knowledgeable CFDC staff and board members, and the strong connection of the CFDC to the community. Other factors which contribute to effectiveness include the high level of accountability, familiarity with local needs, opportunities and stakeholders, and the extent to which the services of the CFDCs are integrated with those of other organizations. Overall, CFDC clients are satisfied with the services they have received, and perceive that CFDC staff are professional, helpful, knowledgeable and responsive, that the services met their needs, and the services (particularly the loans) were not available from other sources.

10. The main factors which constrain the effectiveness of the CFP include internal factors such as the level of funding for CFDC operations, and external factors such as the 2008 economic recession, the challenges associated with the small size of communities, and a lack of willingness of some governments or organizations to partner with the CFDC. The level of funding available to CFDCs constrains their ability to deliver services and dedicate staff time to support community strategic planning, CED projects and partnerships since no funding is set aside for these activities. Other internal constraints include confusion regarding the objectives of the program, the amount of CFDC staff time dedicated to CFP reporting, and a lack of clarity around the roles of the four Community Futures associations. External constraints include economic conditions (e.g., the 2008 economic recession), challenges associated with the small size of communities (e.g., municipal infrastructure), and a lack of willingness among some local governments or organizations to partner with the CFDC (e.g., lack of awareness of the CFDC, more focused on own community's growth instead of regional goals, etc.). CFDCs have implemented various strategies to mitigate internal and external barriers, such as changes to their lending activities and services, increased partnering, and changes to their delivery model, among others.

Performance Measurement

11. While extensive performance information is collected and reported and best practices have been developed (e.g., Business Number analysis using Statistics Canada data), there are issues associated with performance measurement and reporting that need to be addressed, such as the accuracy and utility of some measures, the consistency in reporting across CFDCs, and the level of effort required. FedDev Ontario collects extensive information on a quarterly and annual basis from the CFDCs to meet the requirements of the Performance Measurement Strategy and for planning and reporting purposes. In addition, the Business Number analysis using Statistics Canada data, which compares the performance of CFDC clients and a similar group of businesses, was identified as a best practice in performance measurement in terms of data accuracy, attribution and analysis of impacts. However, there are some significant issues associated with performance measurement that need to be addressed, including:

Performance – Efficiency and Economy

Efficiency

12. The CFP service delivery model is an efficient and effective approach to community economic development. The local and flexible delivery structure allows CFDCs to focus on local needs and build trust with partners and businesses. The CFP service delivery model is efficient and effective because it allows CFDCs to respond to local needs at a grassroots level. Strengths of the model include local delivery, independence from government, the flexible delivery structure, the integrated services (i.e., business financing and counselling, support for CED projects, and support for community strategic planning), extensive use of partnerships, the long history of the CFDCs in the communities, and the strong commitment of the CFDC volunteers and staff.

13. Even if only the impacts generated from services provided to loan clients are considered, the CFP generates a strong return on investment for every dollar invested annually in program grants and contributions. Based on Business Number analysis using Statistics Canada data, for every dollar contributed (based on an average annual CFP grants and contributions budget of $11.8 million), the CFP in southern Ontario generated $43.76 in business revenues and $10.97 in wages, and cost $3,053 per job created in the five year period since loans were received. The costs per job are comparable to other programs with similar objectives (e.g., The CME-SMART Program funded through FedDev Ontario's Prosperity Initiative resulted in an average cost of $7,751 per job created and $3,545 per job maintained). Furthermore, the figures significantly understate the impacts since they do not consider the impacts of services provided to CFDC clients that received only counselling services or training or for the contribution to CED initiatives and community strategic planning activities.

14. There is room to improve the efficiency of the delivery of the CFP in southern Ontario since costs tend to be higher for some outputs in southern Ontario than in Western and Atlantic Canada, efficiency varies widely across CFDCs, and the roles and efficiency measures of the Community Futures associations could be better defined. For some outputs, the delivery of the CFP in southern Ontario is less efficient than Western and Atlantic Canada, particularly in terms of budgeted costs per CFDC staff member and loan issued. However, southern Ontario tends to have lower costs per job created or maintained and leverage more business financing per dollar cost. The differences are due, in part, to differences in the types of services delivered and how data is collected and reported. There is also a significant variation in costs per outcome across southern Ontario CFDCs, which could also indicate varied levels of efficiency or demand for services. In addition, while the CFDCs are generally satisfied with the services provided by the four Community Futures associations, no clear performance indicators are in place to assess the effectiveness, efficiency or economy of their services.

Economy

15. While the past four years have been a transition period for the administration of the CFP in southern Ontario and a number of activities have been undertaken to optimize delivery, the level of funding, new funding model, and governance challenges may constrain the achievement of program outcomes. FedDev Ontario has been working to optimize program outcomes since it took over the administration of the CFP in southern Ontario in 2009 (e.g., holding consultations with CFDCs, developing Key Performance Indicators and a Performance-Based Funding Model, etc.). Though it is too early to determine the impacts of these changes, there is some concern that the reduced level of guaranteed funding and uncertain level of performance-based funding will constrain CFDC delivery of services and may discourage CFDCs from addressing CED and community strategic planning needs in their communities (e.g., only 10% of outcome-based funding pertains to CED and strategic planning performance). Incentives under the new Performance Based Funding Model could also affect loan decisions, influence reporting, impair the ability of CFDCs to attract or retain staff, or add to the complexity of making longer-term strategic decisions. Challenges with respect to governance of the program may also constrain outcomes (e.g., a lack of clarity in program objectives, changing and duplicative reporting, delays in contract approvals and auditing for claims, inflexible deadlines, and too much oversight).

Management of Loan Funds

16. Though there is some variation across individual CFDCs, on average, the CFP loan funds are well managed. Over the four years, 71% of the funds were in active loans and funds grew by 7%. In addition, the lending limits and loan loss rates are appropriate. Over the four year period, an average of 71% of the investment funds in both eastern and western regions were in active loans. The total value of the investment funds grew by 7% (about $12 million) from 2009–10 to 2012–13, as a result of positive returns earned on the loans. Recapitalizations through CAF contributed to an additional increase of $29 million. In addition, key informants perceived that the loan loss rate (average of 4.5%) and lending limit ($150,000 and $250,000, on an exceptional basis) are appropriate. However, there may be opportunities to further optimize the use of loan funds in some regions since some funds were almost completely utilized in active loans, while other funds were only about half utilized (e.g., increase flexibility through regional loan pools to allow CFDCs with high demand for investment funds to access under-utilized investment funds in regions with lower demand).

B. recommendations

The major recommendations arising from the evaluation of the Community Futures Program in southern Ontario are as follows:

1. Pursue opportunities to develop further partnerships and increase collaboration with other organizations in order to leverage CFP services (i.e., business services and lending and support for CED projects and community strategic planning). With a local presence and a flexible structure which allows services and activities to be tailored to local needs and conditions, the CFDCs are well-positioned to facilitate a coordinated and integrated approach to both small business development and community economic development. While recognizing the record of CFDCs in developing strong relationships with other organizations, there are opportunities for CFDCs to further:

FedDev Ontario should work closely with other federal and provincial government programs and services to further coordinate and integrate services. While coordination can and does occur at the local level, there are also opportunities to further coordinate and even consolidate programs and services at the government level to improve CFP effectiveness, efficiency, and economy.

2. Build upon performance measurement efforts by addressing attribution and consistency challenges, reducing duplication, monitoring performance measurement systems, and working with other RDAs to share best practices. FedDev Ontario has an opportunity build upon performance measurement activities and the mechanisms that have been developed to track and measure CFP performance. Specific areas include:

3. Continue to include CED and community strategic planning as core program activities and facilitate CFDC involvement in these areas by supporting CFDCs in leveraging funding from other sources, ensuring the activities are sufficiently resourced, and reducing the reporting burden on CFDC staff time. CFDCs play a key role in providing in-kind and staff support to catalyze and leverage funding for CED and community strategic planning. FedDev Ontario can support CFDC involvement in these areas by encouraging CFDCs to leverage funding from other sources, ensuring the activities are sufficiently resourced (either through Performance Based Funding or other funding mechanisms) while taking into account contextual differences across regions (e.g., access to EODP funding in the Eastern Region), and by streamlining reporting so that CFDC staff have more time to address needs. With respect to the differences in access to project funding across regions, a further assessment of the relative need and impact of such funding should be determined as part of evaluations of other programs administered by FedDev Ontario (e.g., EODP and CAF).

4. More clearly define the roles (e.g., marketing of the CFP, information sharing and support), mandates, and performance measurement requirements for the four Community Futures associations to improve responsiveness and accountability to CFDC needs. Associations play a key role in areas such as professional development, sharing of best practices, coordination and collaboration across CFDCs and with other organizations, and other support services. Associations could also play a role in further enhancing marketing and branding of the CFP across southern Ontario. Though there is no consensus regarding how the associations should be structured and funded, FedDev Ontario should consider different options with respect to funding the Community Futures associations such as reducing the number of associations funded, encouraging amalgamation, or decentralizing the funding decisions to the CFDCs. In addition, clearer performance indicators and measures should be developed to encourage greater responsiveness and accountability. Indicators could include CFDC satisfaction levels, program delivery performance, CFDC member participation levels, and service delivery levels.

5. Strengthen communication and partner relationships with CFDCs. Recognizing that the past four years represent a transition period for the administration of the CFP under FedDev Ontario, the Agency should continue to work to strengthen communication and its partner relationships with individual CFDCs. FedDev Ontario should provide clear and consistent direction with respect to program objectives around CED and community strategic planning, and expectations regarding lending, business counselling, acceptable loan loss rates (e.g., based on national CFP loan loss rates), and appropriate levels of risk. Project officers play an important role in strengthening the partnerships and should have more face-to-face meetings with CFDCs, increase their awareness of CFDC regional needs, and work with CFDC management and Boards to engage local governments and to facilitate other partnerships. FedDev Ontario could further capitalize the local presence and knowledge of CFDCs to leverage other FedDev Ontario programming.

top of page

I. Introduction

A. background

The Federal Economic Development Agency for Southern Ontario (FedDev Ontario) was established in August 2009 in response to the global economic recession. Home to more than 12 million residents (representing 94% of the population in Ontario, and 36% of the total population of Canada), southern Ontario is a key contributor to the health of the Canadian economy. FedDev Ontario was given a five-year mandate to support the competitiveness, innovation, and diversification of southern Ontario's economy by delivering strategic investments to businesses, not-for-profit organizations and communities; establishing and strengthening collaborative partnerships with key economic stakeholders; and representing the region's interests at the federal and national level. This mandate was renewed for another five years in the March 2013 budget.

Operating with an annual budget of $218.8 million and 216 full-time equivalent positions in 2012–13, the Program Alignment Architecture (PAA) of FedDev Ontario consists of four program activities including Business Development; Technological Innovation; Community Economic Development; and Internal Services. As part of the Community Economic Development Program Activity, FedDev Ontario manages the Community Futures Program (CFP) in southern Ontario, which supports 37 Community Futures Development Corporations (CFDCs). Actual spending under the CFP for 2012–13 totalled $12.07 million out of $34.5 million dedicated to the Community Economic Development Program Activity. The budget, sub-activities and key performance indicators for each program activity as well as FedDev Ontario overall are summarized in Appendix I.

B. Purpose of the evaluation

Across Canada, the CFP is delivered through the four regional development agencies (RDAs) in their respective regions as well as by the Federal Economic Development Initiative for Northern Ontario (FedNor) under Industry Canada (IC). In addition to FedDev Ontario, the RDAs include the Atlantic Canada Opportunities Agency (ACOA), the Economic Development Agency of Canada for the Regions of Quebec (CED-Q), and Western Economic Diversification Canada (WD).

The objective of this evaluation is to report on the relevance and performance of the CFP delivered in southern Ontario by FedDev Ontario, in accordance with the 2009 TBS Policy on Evaluation, Directive and Standards. The scope of the evaluation includes CFP activities funded from 2009–10 to 2012–13. Projects that were initiated prior to 2009–10 were the responsibility of FedNor (Industry Canada). The evaluation covers four core areas of the program, as per the 2009 Evaluation Policy. The evaluation issues and questions addressed in this evaluation are provided in Appendix II.

The evaluation approach is based on the Performance Measurement Strategy (PMS) approved for the National CFP. Each partnering department/agency is required to complete an evaluation of their CFP by June 2014.

C. structure of report

The report is structured as follows: Chapter II describes the evaluation methodology, limitations and mitigation strategies; Chapter III provides an overview of the CFP in southern Ontario; Chapter IV, V and VI summarize the major evaluation findings with Chapter IV focusing on relevance, Chapter V focusing on performance – effectiveness, and Chapter VI focusing on performance – efficiency and economy; and Chapter VII summarizes the major conclusions and recommendations.

II. Evaluation methodology

A. evaluation approach and methodology

The evaluation used multiple lines of evidence including primary data and secondary data sources.Footnote 2 Specifically, the data collection included:

Table 1: Communities Selected for Case Studies
CFDC Topic of the Case Study
Trenval Business Development Corporation The purpose of this case study was to examine the co-location partnership model Trenval has developed with other economic/business development organizations, and how this model enhances coordination and support for development in the region.
Frontenac Community Futures Development Corporation The purpose of this case study was to learn about Frontenac CFDC’s food and beverage promotion activities and how the activities have benefited business and economic development in the region.
Nottawasaga Futures The purpose of the case study was to examine the Green Economy Centre and how the Centre has supported businesses to reduce costs, maintain jobs and grow.
Sarnia–Lambton Business Development Corporation The purpose of the case study was to assess the Aboriginal Youth Entrepreneurship Program and how it has helped to increase awareness of entrepreneurship, and build business skills among Aboriginal youth participants.

The case studies included document reviews, interviews, focus groups in two communities to which site visits were made, and an analysis of surveys or interviews already conducted with representatives in the communities selected.

The evaluation focuses most heavily on the CFDC services provided to businesses, particularly the provision of loans which tend to have the most direct and measurable impacts. The CFDCs are also active in community economic development and strategic planning activities. However, most funding provided by FedDev Ontario for such projects is delivered through other programs and, as such, the results of those projects will be assessed primarily as part of other evaluations.

B. Challenges and limitations

The challenges and limitations associated with this evaluation included the potential for respondent bias and non-response error. Many key informants and clients are involved in program design and delivery or are direct beneficiaries. Consequently, the findings may be biased towards more favourable program outcomes. Several measures were taken to reduce the effect of respondent biases: (i) the purpose of the survey or interview and strict confidentiality of responses was communicated clearly to participants; (ii) answers were cross-checked with those of other groups for consistency and validation; and (iii) wherever possible, the findings were triangulated and validated with other lines of evidence.

Factors that contributed to the potential for non-response error among clients included issues associated with population list, the length of time since the services were provided to the clients, and differences in the level of service provided and outcomes. To mitigate the potential effects, three email reminders and follow-up phone calls were made with a particular emphasis on under-represented regions. To assess the non-response error, the characteristics of those surveyed were compared with clients in the sampling frame to assess consistency based on the data currently available. The characteristics of respondents varied from those of the total population in terms of gender, incidence of loan clients, and average size of loan. No adjustments were made to reflect gender as it is believed that the difference relates primarily to a classification issue in the sample lists. The response rate was significantly higher amongst loan clients than non-loan clients because greater emphasis was placed on loan clients; to address this issue, the responses of loan clients and non-loan clients are reported separately. The average value of the loan received per client surveyed was $82,553 as compared to an average of $66,620 amongst loan clients in the contact lists; the difference in the average size of loans is addressed by reporting impacts per dollar of loan funding provided rather than per loan provided.

III. Community futures in southern ontario

This chapter provides a detailed description of the Community Futures Program (CFP).

A. Structure of the program

The CFP is a national program that was established in 1985 and has been administered in southern Ontario by FedDev Ontario since 2009. The objective of the CFP is to promote economic stability, growth and job creation leading to more diversified and competitive rural economies, and economically sustainable communities. The program is delivered through a network of 37 CFDCs providing geographic coverage to rural southern Ontario. CFDCs are independent, community-based, legally incorporated not-for-profit community development organizations that have been established for the purpose of delivering the CFP in a designated service area. All CFDCs have corporate bylaws and have developed policies and procedures for administration of their general operations, personnel and investment fund as well as conflict of interest guidelines. On average, the CFDCs have been in operation for 22 years. Of the 37 CFDCs, six are classified bilingual and two are classified as Aboriginal.

B. The program model

1. Program Activities

As indicated in the program logic model in Appendix V, FedDev Ontario’s activities can be grouped under two categories: development/maintenance and program delivery. Development/maintenance activities include program development, planning and management; monitoring and providing non-financial support to CFDCs; and measuring CFDC performance and allocating funding.

To support program delivery, FedDev Ontario provides financial support to CFDCs through non-repayable contribution agreements. Deliverables specified in the contribution agreement are negotiated based on a business plan developed by the CFDC that includes measurable performance goals, objectives, activity targets and budget forecasts. FedDev Ontario Program officers monitor CFDC performance with respect to the terms and conditions of the agreement and act as a resource to assist the CFDC in carrying out the activities and obligations specified in the agreement. From 2009/10 to 2011–12, the 37 CFDCs in southern Ontario each received $300,000 in annual contribution funding to support CFP activities. In 2012–13, CFDCs each received $285,000 in base contribution funding, as a result of a reduced overall program budget, with 32 CFDCs receiving an additional $5,000 in Best Practices Standards funding as well as additional funding for satellite offices and official languages. The level of contribution funding to CFDCs will change in 2013–/14 as a larger proportion of funding will be determined based on CFDC performance, which is discussed in Chapter VI of this report.

FedDev Ontario, through the CFP, also funds other organizations and activities such as Community Futures associations (e.g., regional, provincial, national) and other activities that support the delivery of the program in southern Ontario. In addition to the national Community Futures Network of Canada, there are three CFDC associations based in Ontario including the Ontario Association of CFDCs (OACFDC), the Western Ontario CFDC Association, and the Eastern Ontario CFDC Network (EOCFDC Network). FedDev Ontario currently provides operational funding to three of the associations and the annual funding allocation is determined by the number of CFDCs they serve. Each association receives $5,000 per CFDC (i.e., $185,000 is allocated to the OACFDC, $75,000 is allocated to the EOCFDC Network, and $110,000 is allocated to the Western Ontario CFDC Association).

2. Outputs

The logic model for the CFP makes a distinction between the outputs of FedDev Ontario and those of the CFDCs. In addition to making funding adjustments and providing contributions to CFDCs to support community economic development, FedDev Ontario also produces policies, plans, reports, program tools, and communications resources (e.g., tools that support the development of effective business practices) and provides advice, information, monitoring and support for the CFDCs. The support provided by FedDev Ontario enables the CFDCs to provide:

3. Intended Outcomes

In the immediate-term, the outputs of the CFDCs are expected to result in improved access to capital and leveraged capital through loans, loan guarantees and equity investments to businesses and social enterprises; enhanced/maintained business development services (e.g., information, counselling, referrals, and training); strengthened community strategic plans and more effective implementation of CED through projects, partnerships and other CED initiatives.

In the intermediate term (e.g., within two to three years), the activities and outputs of the CFDCs are intended to improve business practices and increase entrepreneurship, strengthen and expand businesses, and strengthen community capacity for socio-economic development. In turn, the creation of new businesses and the expansion of existing businesses, improved business practices, increased entrepreneurship, job creation and strengthened community capacity are expected to lead to more diversified and competitive local rural economies, greater economic stability, growth and job creation, and economically sustainable communities.

C. resources

FedDev Ontario receives A-base funding annually to provide financial support to CFDCs. The table below provides estimated and actual financial resources allocated to the CFP in southern Ontario over the period covered under the evaluation. As demonstrated in the table, approximately $50.31 million has been spent on the activities of the program by FedDev Ontario from 2009–10 to 2012–13.Footnote 4

Table 2: CFP expenditures
Fiscal years Actual Expenditures ($ Millions)
Grants & Contributions Operations & Maintenance (approximate) Statistics Canada Business Number Analysis Total
2009–10 $10.84 $0.77 - $11.61
2010–11 $12.54 $0.77 - $13.31
2011–12 $12.54 $0.77 $0.008 $13.32
2012–13 $11.29 $0.77 $0.013 $12.07
Total $47.21 $3.08 $0.02 $50.31

There are 10.45 full-time equivalents (FTEs) allocated by FedDev Ontario to the delivery of the CFP in southern Ontario. Staff split their time between the delivery of the CFP and other CED programs administered by FedDev Ontario. The staff breakdown across each region and office includes 4.75 FTEs dedicated to the Western Region, 3.15 FTEs dedicated to the Eastern Region, and 2.55 FTEs dedicated to the CFP from the Agency’s head office in Kitchener, Ontario.

D. stakeholders

As representatives of their respective communities, the primary stakeholders under the CFP are the CFDCs and Community Futures associations. FedDev Ontario consults with CFDCs on an ongoing basis. CFDCs also collaborate directly with provincial and municipal governments as well as other departments or agencies of the federal government (CFDCs have been used as an enabling mechanism for other federal and provincial government programs and services, both current and past, that advance local and regional development initiatives). The ultimate beneficiaries of the CFP are the businesses and communities in the regions they serve.

top of page

IV. Relevance of the program

This chapter summarizes the major findings with respect to the need for the program, relationship with other programming, and alignment with departmental and federal priorities, roles and responsibilities.

A variety of best practices, effective strategies and opportunities for improvement were identified through interviews with key informants, the survey of clients, case studies and a review of literature. These best practices and areas for improvement are highlighted throughout the report.

A. Continued Need for the CFP

The results of the evaluation demonstrate that there continues to be a significant need for the outputs generated by the CFDCs including business financing, business information and services, community strategic planning, and CED projects as described below:

Communities outside of the major urban areas face significant economic challenges associated with the small population base, limited access to workers (particularly skilled workers), heavy reliance on the natural resource sectors, and limited access to municipal resources.

A 2009 report by the Federation of Canadian Municipalities titled, Wake-Up Call: the National Vision and Voice We Need for Rural Canada: the Federal Role in Rural Sustainability, explains that rural Canadian communities face unique barriers to economic growth and development. The report argues that rural Canada is in crisis and that, "with shrinking tax bases, limited revenue sources and rapidly aging infrastructure, rural municipalities are struggling to provide the basic services and community facilities their communities need to attract and retain residents and businesses."Footnote 5 Furthermore, the report contends that despite growing urbanization, these smaller communities remain critical to the economic, social and environmental fabric of Canada.

Major challenges facing smaller communities in southern Ontario which are served by CFDCs include:

There is a significant need for support targeted at SMEs in smaller centres given the importance of small businesses to rural economies, the aging of the entrepreneurial base, their difficulties in accessing financing, and the need for training, guidance and other support.

SMEs are a major driver of economic activities in Canada, accounting for over 30% of Canada's GDP.Footnote 12 According to Statistics Canada, there were over 400,000 SMEs in Ontario in 2013, representing 98% of employer businesses in the province. Of those, over half have only 1 to 4 employees.Footnote 13 Reliance on the small business sector is even greater in rural communities; in 2007, 67% of businesses in smaller centres had 1 to 4 employees, compared to 56% of businesses in large urban centres.Footnote 14

The entrepreneurial base in these smaller communities is aging and will need to be replaced. For example, according to a recent Statistics Canada publication,Footnote 15 self-employed people (excluding farm jobs) accounted for about 17% of total employment in rural and small town Canada (as compared to 14% in the larger urban centres). Of the self-employed workers in the smaller centres, 24% are currently aged 55 to 64 years and will likely retire within 10 years.

Financing is critical to the establishment and development of businesses. According to Industry Canada 36% of small businesses requested external financing in 2011 with 26% requesting debt, 7% requesting leasing, 8% requesting trade credit, 4% requesting government financing and 2% requesting equity financing.Footnote 16 Literature indicates that SMEs are experiencing increasing challenges in obtaining financing due to more restrictive financing terms, higher costs of credit, and collateral requirements. Approximately 60% of businesses found that their banks have tightened some of the financing terms (e.g., increased the cost of credit or levels of security required).Footnote 17

Smaller businesses, particularly those with fewer than five employees, are least likely to apply for financing and be approved (for example, in 2010, only 19% of SMEs in Canada with fewer than five employees applied for external financing).Footnote 18 Smaller businesses are at a particular disadvantage when seeking financing as the process demands many hours of management's time and they have limited time and resources available to meet those demands; start-up and recently established businesses carry the greatest risks for creditors and therefore have greatest difficulties in securing financing because they have an unproven track record and are more likely to have short credit lives and minimal tangible assets;Footnote 19 and, for financing providers, smaller investments can cost as much to assess and manage as larger ones, making the economics of small business financing precarious.Footnote 20 As such CFDCs can play an important role in providing lower value loans to small businesses (the average loan value approved by CFDCs in 2010 was $46,000 which is less than one-sixth of the average debt financing of $275,000 approved for all SMEs in Ontario in 2010).Footnote 21

Many SMEs also need assistance other than financing to be successful. Most entrepreneurs served by the CFDCs do not have formal education in business management (of the clients who were surveyed as part of this evaluation, only 27% possessed a college diploma or university degree related to business management) and most had little business management experience prior to approaching the CFDC. Most clients indicated that they had little business management experience when they were first assisted by the CFDCs. According to a literature review which examined entrepreneurship education and training in Canada, "training and education of Canadian entrepreneurs have been repeatedly cited as an effective way to reduce small business failure in Canada" (Ibrahim and Ellis, 2002; Menzies and Gasse, 1999; Carrier, 1999; Béchard and Toulouse, 1997).Footnote 22

There is also a need for organizations to support community strategic planning and economic development efforts, liaise between neighboring local governments, and catalyze the implementation of CED initiatives and regional economic development plans, particularly in communities without a strong leader in economic development.

Providing support directly to small businesses is an important strategy in promoting community economic development in smaller centres. However, there are some issues such as business retention and attraction, labour force development, and infrastructure development that are best addressed through broader systemic initiatives and in partnership with local governments. For example, a note by the Rural Ontario Institute (2012) recommends that, in its programming, government should not focus only on providing direct support for business:

"Many important areas of economic development programming are beyond the purview of an individual private sector business. Initiatives around, for example, labour force development, immigration attraction, industry-education partnerships, manufacturing networks, business incubators and the like are all types of initiatives which are likely to [promote economic development]. In dialogues with rural municipal leaders and through emerging initiatives surrounding rural social enterprise, we hear many ideas for projects that would create the conditions for economic success and the opportunity to catalyze such initiatives through municipal or non-profit groups should not be overlooked."Footnote 23

Local governments, particularly in smaller communities, are in need of support for community strategic planning and economic development activities, in part, due to their own limited resources. Smaller communities often have no municipal economic development officer (EDO) or only a part-time officer undertaking economic development activities for the community. According to a detailed online search of municipal government websites, only 41% (99 out of 241) of the municipalities located within southern Ontario CFDC regions have at least one contact person assigned to economic development. There is not a significant difference between the Eastern and Western Regions; 44 percent (47 out of 107) of municipalities in the East compared to 39% (52 out of 134) of municipalities in the West have a dedicated EDO on staff.

CFDCs are often seen as vital partners of local governments in providing economic and research data support, providing coordination and liaison support (between local governments and different levels of government), and acting as conduits through which communities can access grant funding for CED (e.g., federal, provincial, or private funding).

Spotlight on Best Practices and Opportunities for Improvement–The CFP supports CFDCs in catalyzing regional partners and leveraging funding to undertake CED and community strategic planning: The case studies demonstrated the critical role that CFDCs play in catalyzing CED and strategic planning by bringing together regional partners and leveraging funding from other sources:

Most key informants participating in the evaluation reported a strong need for the programming and services provided through the CFP to help address challenges faced by SMEs and to support rural economies through building capacity, increasing diversification, and slowing outmigration.

When asked to rate the level of need on a scale of 1 to 5, where 1 is no need and 5 is major need, the average ratings ranged from 4.5 amongst FedDev Ontario representatives to 4.7 amongst other stakeholders and 4.9 amongst CFDC representatives and community partners. Key informants confirmed the major challenges identified in the literature, noting that the CFP is needed to:

A few stakeholders noted that, in some communities, there may be less of a need for the CFP because other organizations (e.g., municipal or regional governments) take the lead on strategic planning and CED functions or are meeting the need for small business services, particularly financing.

Both CFDC loan clients and CFDC clients that received services other than loans indicated a strong need for the type of services provided by the Community Futures Program.

When asked to rate the need for an organization such as the CFDC, on a scale of 1 to 5 where 1 is no need and 5 is a major need, loan clients provided an average rating of 4.6 while non-loan clients provided an average rating of 4.4. Nearly one-half of clients referred to the important support that CFDCs provide to new entrepreneurs and small businesses including guidance, mentoring, education, and opportunities for networking not provided by other organizations or private lenders. One-quarter of clients highlighted the important support CFDCs provide to small, start-up or innovative businesses and entrepreneurs who do not have much experience or training but have viable business ideas. Some clients who rated the need for services as less important reported that CFDC services, particularly loans, are not sufficient to support growing businesses and can be accessed from other sources.

B. Relationship to other programming

Other programs are available to businesses and rural communities in southern Ontario which share objectives similar to those of the CFP. These programs can be divided into three categories: programs which provide financing for small business; programs which provide other services for SMEs; and programs which support community strategic planning and CED initiatives. As described below, the CFP tends to complement rather than duplicate or overlap these other programs.

According to most key informants, the potential for overlap or duplication is reduced by the high level of coordination, collaboration, leveraging of resources and differentiation between programs. The main areas of potential overlap are in the provision of basic business information and services as well as large loans (i.e., in the upper end of CFDC lending amounts).

Of the key informants who were interviewed (n=162), 74% indicated that there is no overlap or duplication between the CFP and other programs while 24% noted some degree of duplication with respect to specific components of the CFP. However, only one key informant indicated that the duplication was significant.

Key informants who perceived there was little to no duplication explained that there is no other organization like the CFDC, providing the same range of services (i.e., SME financing and counseling, and support for CED and community strategic planning) with local decision-making in rural areas of southern Ontario. The key informants noted most commonly that:

The potential for duplication is reduced by the high level of collaboration and coordination between CFDCs and other service providers. For example, representatives from five CFDCs indicated that their CFDC has co-financed loans with the BDC. In one specific transaction, for example, the BDC provided $200,000 in financing for an SME to purchase land and a new building, and a CFDC provided $50,000 to finance working capital. The financing in this particular deal led to the creation of 17 new jobs in the region.Footnote 24

CFDCs also reduce the potential for overlap in business services through regular communication with other service providers, mutual client referrals and, in some cases, the co-delivery or co-location of services with other similar programs. The potential areas of duplication or overlap that were most commonly identified were in the provision of business counseling, information, and training, and, to some extent, business financing. Several key informants (25) argued that CFDC business counseling services may overlap somewhat with the provincially and municipally funded Small Business Enterprise Centres. Some key informants (10) noted that the potential for overlap between the CFDCs and the BDC increases as the size of the average CFDC loan increases. Other potential areas of duplication noted included private sector financiers such as banks and credit unions (3), programs that fund CED activities (3), and community strategic planning conducted by municipalities (3).

The CFP fills a significant gap in the provision of financing for SMEs in smaller centres, particularly with respect to lower value loans. Most loan clients indicated they would not have been able to obtain loans elsewhere.

Other than commercial financing, the most commonly mentioned sources of financing which are similar to the CFP included the BDC, the Canada Youth Business Foundation (CYBF), the provincially funded Self-Employment Benefit Program, the Southern Ontario Fund for Innovation and Investment (SOFII), and Aboriginal Business Development Canada (ABDC).

Although the CFP can overlap to some extent with BDC financing for high value loans (e.g., $250,000), the two organizations often work together to finance the same client and find mutual benefits to sharing risk. CFDCs bring specific knowledge of the local region and character of the borrower, and can finance clients and costs that BDC is less willing to finance (e.g., start-up businesses, working capital costs).

Other programs do not overlap much with the CFP since they are limited to specific target groups or sectors (e.g., Farm Credit Canada focuses on agriculture-related businesses). In addition, CFDCs collaborate and coordinate with programs (e.g., CFDCs sit on the board for SOFII) to reduce the potential for duplication, leverage available financing, and refer clients where appropriate.

The results of client survey suggest that the CFP is filling an important gap with respect to financing for SMEs. The 218 loan clients who received a loan in the past four years were asked whether they had approached another program or financial institution for funding prior to applying to the CFDC and how likely it is they would have been able to obtain their loans from another source (e.g., from banks or other financial institutions) if the CFP had not been available. The results indicate that 59% of the loan recipients had approached another program or financial institution for funding prior to applying to Community Futures (55% approached a commercial lender, 7% approached a federal government program, 2% approached a provincial government program, and 1% approached private lenders) of whom 37% were approved for the loan while 63% were not approved. Of those who were approved loans, two-thirds obtained funding from both the CFDC and the other source while one-third decided to take the loan from the CFDC but not from the other source because the interest rate, the equity and collateral requirements, and the repayment terms were considered to be better. The primary reasons why the applicants were turned down by the other source included concerns about collateral (44% of those who were not approved), the recent financial performance of the business (19%), credit history (16%), the manager's limited experience in business (15%), and concerns about the local economy (15%). None of the applicants identified the lack of a business plan or concerns about the business plan as the reason for not being approved. The loan clients who did not approach any other sources, on average, estimated that there was a 44% likelihood that they would have been able to obtain the loan funds from another source (e.g., from banks or other financial institutions) if funding from the CFDC had not been available.

The CFP reduces the potential for duplication with other programs that provide business information through collaboration, partnership and focusing on smaller centres and more advanced SMEs.

The most commonly mentioned programs which provide similar support services to businesses as the CFP included the provincially funded Small Business Enterprise Centres (SBECs),Footnote 25 Self-Employment Benefit Program, Regional Innovation Centres (RICs),Footnote 26 and Business Advisory Services (BAS), as well as BDC's consulting services, CYBF mentoring, and the local Chambers of Commerce.

The greatest potential for overlap is in the provision of basic information for early stage and start-up SMEs. The SBECs, the Self-Employment Benefit Program, Canada Business Ontario, and the CYBF all provide some kind of basic information on how to start a business. However, the programs mostly complement the CFP because each has a slightly different target group and can help SME clients at different stages of development. For example, individuals with no experience or training in business can use services such as the Self-Employment Benefit Program (if eligible) and SBECs. Then, once individuals have a business idea or plan, they can approach a CFDC to refine their plan and apply for financing. If the business is successful and grows, the BDC or the provincial government's Business Advisory Services can assist them with more strategic advice. Many CFDCs recognize these opportunities to leverage other similar services, and often collaborate with other programs in providing referrals, delivering joint-workshops, co-sponsoring events, co-locating, and in some cases, delivering the programs themselves (e.g., SBECs and the Self-Employment Benefit Program).

The Government of Ontario is currently undertaking a review to identify overlaps or gaps in the services it delivers, and to assess the extent to which various services work together to serve the region. In addition, as part of the CFP transformation, FedDev Ontario has placed an emphasis on enhanced collaboration within and outside the CFDC network. For example, FedDev Ontario will help facilitate pilot projects to align CFDCs and SBECs more closely to identify cost and reporting synergies.Footnote 27

Only 19% of clients (both loan and non-loan) reported receiving assistance, other than financing, from programs or agencies other than CFDCs that significantly helped them in establishing and developing their business. The most common types of assistance included assistance in preparing a business plan, business counselling and referrals to other resources.

Table 3: Number and Percentage of Loan and Non-Loan Clients Reporting Significant Assistance Other Than Financing from Another Program (Types of Assistance)
Organization or Program Loan Clients Non-loan
Number % Number %
Number Reporting Significant Assistance Other Than Financing 56 100% 39 100%
Assistance in the development of a business plan 22 39% 15 38%
Business counselling 22 39% 14 36%
Referrals to other resources 16 29% 19 49%
Participating in a training program or seminar 15 27% 13 33%
Access to business information 14 25% 13 33%
Participation in a mentoring program 6 11% 6 15%

The other organizations or programs most commonly identified as providing CFDC clients with this assistance included federal and provincial government departments (20% of clientsSBECs (16%), chambers of commerce (12%), BDC (9%), colleges (9%), industry groups/associations, centres (9%), employment/training centres (7%), CYBF (5%), Self-Employment Programs (5%), local business advisory centres (4%), professional services (3%), and banks (3%).

The CFP complements other resources by providing leadership, local knowledge, expertise, and support to strengthen CED projects and municipally-led community strategic planning.

The programs mentioned most frequently as supporting CED and community strategic planning included the federally funded Eastern Ontario Development Program (EODP) and Southern Ontario Development Program (SODP), the provincially funded Eastern and Southwestern Ontario Development Fund (EODF/SODF), Communities In Transition Initiative (CIT), and Rural Economic Development Program (RED), as well as local government economic development offices and private funders such as the Trillium Foundation. The various resources complement each other by coming together to work on specific initiatives and leverage funding and other support. For example, eastern CFDCs can use EODP funds to support CED projects. In addition, provincially funded programs such as the EODF, SODF, CIT, and RED can stack with funding from CFDCs. It was also noted in key informant interviews that CFDCs have partnered with provincial programs by providing in-kind support (e.g., client engagement) and knowledge of local community needs. The only area where there might be overlap is in the use of local volunteers to participate in project committees.

With respect to community strategic planning, as some rural communities grow, there is less of a need for CFDCs to provide capacity building support or to lead community economic planning since many communities are undertaking their own planning activities. CFDCs can, however, continue to assist municipalities to undertake CED (either through financial or in-kind support). CFDCs also have a role to play in bringing municipalities together to develop regional strategic plans. CFDCs also maintain a close collaboration with municipal representatives to align their activities with community strategic priorities.

C. Alignment with federal priorities, roles & responsibilities

The results of the evaluation demonstrate that the CFP is well-aligned with the priorities of FedDev Ontario and the priorities, roles and responsibilities and the federal government as described below:

The objectives of the CFP are aligned with strategic outcomes of FedDev Ontario related to economic stability and growth of communities, as well as with federal priorities and strategies with respect to economic growth and jobs.

According to the Community Futures Program National Performance Measurement Strategy (2013), the CFP supports community economic development and builds the self-reliance and capacity of communities to realize their full sustainable potential. The objectives of the CFP in support of this mandate include: 1) fostering economic stability, growth and job creation; 2) helping create diversified and competitive local rural economies; and 3) helping to build economically sustainable communities.Footnote 28

The CFP objectives align well with the expected results of the CED program activity and the strategic outcomes of FedDev Ontario. The expected result of the CFP is that "southern Ontario communities have strong economies able to sustain long-term economic development and growth," which contributes to the expected result of the Community Futures sub-program that "rural communities in southern Ontario have strong economies to help respond to future economic challenges." The expected results of both the sub-program and the Community Economic Development program contribute to the FedDev Ontario's ultimate strategic outcome: "a competitive southern Ontario economy."Footnote 29

The objectives of the CFP also align with federal priorities and strategies. The Agency's CED program aligns with the Government of Canada's outcome of "Strong economic growth: Program activities aim to create an environment conducive to economic growth and to promote the development of all sectors of the economy and in all regions of Canada." Recent Speeches from the Throne (October 2013, June 2011, and March 2010) echo the federal government's commitment to job creation, support to small businesses, and economic well-being of communities, noting that, "creating jobs and securing economic growth is and will remain our Government's top priority,"Footnote 30 and that, "the government can help create the conditions for communities–and the industries that sustain them–to succeed,"Footnote 31 and that "small- and medium-sized businesses are the engines of the Canadian economy, responsible for the creation of most new jobs."Footnote 32 The CFP is aligned with Government of Canada priorities; the emphasis on entrepreneurism, access to capital for small businesses, and knowledge and skills development, all of which improve Canada's overall productivity, competitiveness, and growth is well-aligned with Advantage Canada.Footnote 33

The objectives of the CFP are aligned with the commitment of the federal government, under the Constitution Act (1982), to promote regional economic development.

The federal and provincial governments share powers and responsibilities related to regional economic development. Although the federal government has long been involved in regional economic development, it was only with the passing of the Constitution Act, 1982 that, under Section 36, the Government of Canada and the provincial governments committed, amongst other responsibilities, "to furthering the economic development to reduce disparity in opportunities."

To help the region mitigate and overcome regional and global economic challenges, the Government of Canada created FedDev Ontario in 2009. To fulfill its mandate, FedDev Ontario supports community economic development, business development, and technological innovation in southern Ontario by working with partners and stakeholders across the region and responding to emerging opportunities and challenges. The Agency directs resources toward activities with the greatest impact to address community and business needs, and as a means of propelling the southern Ontario economy forward to a position of strength and prosperity, both in Canada and on the global stage. FedDev Ontario is responsible for positioning southern Ontario as a key factor in building a stronger Canadian economy.Footnote 34

The objectives of the CFP are aligned with the activities, roles and responsibilities of FedDev Ontario. The CFP was established in 1985 with the objective of mitigating the labour market adjustment needs in non-metropolitan areas experiencing chronic or acute unemployment. The CFP provides a flexible, national resource to support a bottom-up and participatory approach to community adjustment, diversification and development. As the program has evolved and grown in support of Community Futures organizations across the country, it continues to support rural communities and SMEs in meeting their economic needs, developing and implementing long-term community strategic plans leading to the sustainable development of their local economies and by providing resources to local CFDCs to build community capacity to adapt to and manage change.Footnote 35

top of page

V. Performance - effectiveness

This chapter summarizes the key findings of the evaluation regarding the outputs and outcomes of CFP activities, factors contributing to and constraining effectiveness, and performance measurement.

A. Financing and business development services

The results of the evaluation indicate that the CFP has provided business financing and services to large numbers of SMEs in southern Ontario, which has leveraged sources of funding, improved business practices, and strengthened and expanded local businesses.

1. Outputs and Immediate Impacts

From 2009–10 to 2012–13, CFDCs issued 2,779 new loans valued at over $165 million. A greater number and value of loans were issued per CFDC by CFDCs in the East compared to those in the West, which can be explained, in part, by differences in population density and outliers.

As shown in the table below, CFDCs issued 2,779 new loans ($165 million total) from 2009–10 to 2012–13. The highest number of loans was issued in 2009–10 (813 loans), which may reflect greater demand for loans as a result of tighter credit conditions introduced by financial institutions following the 2008 economic recession.

Table 4: Summary of New and Active Loans by Year
Year Number of CFDCs Number of Loans Value of Loans Average Size of Loans
New Active New Active New Active
By Year (Total)
2009–10 37 813 2,349 $48,020,401 $123,774,918 $59,066 $52,693
2010–11 37 704 2,561 $46,261,412 $138,930,723 $65,712 $54,249
2011–12 37 557 2,485 $28,840,343 $134,431,546 $51,778 $54,097
2012–13 37 705 2,619 $42,239,755 $140,410,002 $59,915 $53,612
Total 37 2,779 -- $165,361,911 -- $59,504 $53,680

CFDCs in the Eastern Region, on average, issued a higher number (105 vs. 55 per CFDC) and value ($5.6 million vs. $3.7 million per CFDC) of loans than did CFDCs in the West, as shown in the table below. This difference can, in part, be explained by differences in population density. The region occupied by Western CFDCs has a 55% higher population density than the region occupied by Eastern CFDCs (31.5 persons per km2 in the West vs. 20.3 persons per km2 in the East in 2011). Since CFDCs in the Eastern Region have a lower population density, they also typically have fewer business financing and counselling services available, which can explain a relatively higher demand for CFDC services. Part of the difference can be explained by major outliers in the data. If we remove major outliers, the East-West difference is significantly reduced (82 vs. 55 per CFDC in terms of the number of loans issued and $4.7 vs. $3.7 million per CFDC in terms of the value of loans issued). Another reason for the difference could be a lack of awareness or sufficient marketing of the services offered by the CFDCs.

Table 5: summary of new loans by CFDC Region, 2009–10 - 2012–13
Year Number of CFDCs Number of
New Loans
Value of
New Loans
Average Size of
New Loans
By CFDC Region (Total, 2009–10 - 2012–13)
East 15 1,576 $84,293,024 $53,485
West 22 1,203 $81,068,887 $67,478
Total 37 2,779 $165,361,911 $59,504
By CFDC Region (Average Per CFDC, 2009–10 - 2012–13)
East -- 105 $5,619,535 $53,485
West -- 55 $3,684,949 $67,478
Total -- 75 $4,469,241 $59,504

According to administrative data, CFDCs received 138,379 general inquiries and provided 37,147 in-depth interviews. Examples of services delivered by CFDCs include workshops, seminars, events networking opportunities, referrals to other programs and services, and programs to enhance business knowledge and skills among youth.

CFDCs provided a range of business information and services such as on-site libraries and guided access to internet-based information, and referrals to other services and specialists as required. In-depth interviews are defined as one-on-one counselling with individual clients to support the start-up, expansion and diversification of small businesses and to improve their competitiveness.Footnote 36 The counselling can include, for example, assistance in the preparation of a business plan, marketing, accounting or referrals to other programs or support. The reported number of general inquiries and in-depth interviews has varied over the years and across individual CFDCs. For all four years, the number of general inquiries reported annually per CFDC ranged from 150 to over 18,000 (11 CFDCs reported receiving under 1,000 inquiries and 4 CFDCs reported receiving over 10,000 inquiries). The number of in-depth interviews reported annually per CFDC per year varied from 174 to over 4,900 (12 CFDCs reported under 500 in-depth interviews and 5 reported over 2,000 in-depth interviews). There was not much difference between CFDCs in the East and West in the average number of in-depth interviews provided per CFDC (1,021 in the East vs. 993 in the West). However CFDCs in the West received about 70% more inquiries than CFDCs in the East (4,468 in the West vs. 2,672 in the East).

We reviewed a sample of 61 annual reports submitted by CFDCs, which reported on the number of workshops, seminars, and lunch & learns or breakfast meetings delivered. On average, the CFDCs reported annually delivering an average of 3.5 seminars, 3.0 lunch & learns or breakfast meetings, 2.9 workshops, and 1.2 other events or sessions to a total of 151 participants. The services provided included workshops and training in specific topics such as marketing, social media, branding, organizational development, human resources, financial literacy, taxation and legal considerations, business management, importing and exporting, and succession planning; some workshops were offered in partnership with other services providers (e.g., the Business Development Bank of Canada and the Small Business Enterprise Centres).

Spotlight on Best Practices and Opportunities for Improvement – The CFP supports CFDCs in enhancing knowledge, skills and awareness of entrepreneurship among local youth: The case study of SLBDC's Aboriginal Youth Entrepreneurship Program provides an example of CFDC services which enhance knowledge, skills and awareness of entrepreneurship. The project was developed in partnership between the CFDC, two First Nations, a college, and Tecumseh CFDC. The goal of the project was to provide entrepreneurial education and training to Aboriginal youth. The youth attended a "Bootcamp" style training session at the CFDC and then were taken through the process of identifying, researching, documenting and refining a business concept. Over 100 hours were spent working with six youth over a period of approximately six weeks. The program culminated with the students developing a business plan and presenting their business concepts.

The clients surveyed reported receiving a variety of services from their respective CFDCs.

Over one-half of clients (58%) reported receiving more than one type of service from their regional CFDC. The figure below summarizes the percentage of loan clients and non-loan clients who reported receiving various types of services.

Figure 1: Types of Services Received by CFDC Clients Surveyed (N=512)

Graph of Types of Services Received by CFDC Clients Surveyed (N=512) (the long description is located below the image)
Description of Figure 1

Figure 1 is a line graph chart illustrating the Types of Services Received by CFDC Clients Surveyed.

It reflects two different clients including loan clients and non-loan clients.

The chart shows specific services received by a percentage of loan clients:

  • 100% receive a loan
  • 54% receive business counselling (one-on-one)
  • 42% participated in a training program or seminar
  • 41% received assistance in the development of a business plan
  • 35% received business information
  • 26% used pathfinding to identify other business resources
  • 13% participated in a mentoring program

The chart shows specific services received by a percentage of non-loan clients:

  • 0% receive a loan
  • 29% receive business counselling (one-on-one)
  • 27% participated in a training program or seminar
  • 29% received assistance in the development of a business plan
  • 27% received business information
  • 16% used pathfinding to identify other business resources
  • 9% participated in a mentoring program

The pathfinding services involved referrals to government programs (identified by 20 clients), business or government services such as training or business registration services (19), professional services (16), and other sources of funding. The clients received services an average of nine times each over the past four years. The median number of contacts over the past four years was three times amongst both loan clients and non-loan clients.

Key informants perceived that CFDCs were successful in contributing to improved access to capital and enhanced business services (ratings ranged from 4.1 to 4.5).

When asked to rate how successful they believe the CFDC has been in facilitating improved access to capital and enhanced business services, key informants (n=158), including CFDC representatives, community partners, and other stakeholders provided average ratings of 4.4, 4.5, and 4.1 respectively, on a scale from 1 to 5, where 1 is not at all successful and 5 is very successful. On average, key informants perceived that there was a 21% likelihood that CFDC clients could have received loans and other services from other programs in the absence of the CFDC. It was noted that in some communities the CFDC is the only available service for businesses (e.g., Aboriginal communities).

The key informants that noted there was some likelihood that clients could have received services from other organizations, explained that clients could have received loans or funding from banks, angel investors, the Business Development Bank of Canada, Aboriginal programs, or friends and family. They also mentioned that clients could have received services and advice from the SBECs and the RICs. Key informants further elaborated that the loans would have been different because banks are less flexible on repayment terms, charge higher interest rates, and do not provide the same level of support or advice to clients. It was also noted that the services and advice received would not have been as in-depth as those provided by the CFDC.

2. Intermediate Impacts

The assistance received by clients had a range of impacts on them, particularly with respect to developing business-related skills. Given that they tend to have less business experience, non-loan clients were more likely to report that the CFDC assistance had a major impact in enabling them to further develop their skills or encouraging them to start their own business.

Using a scale of 1 to 5, where 1 is no impact at all and 5 is major impact, clients were asked to rate the impact that the services they received have had in terms of increasing their access to other programs and services; helping further develop their business, management or personal skills; helping to improve their business practices; and encouraging them to start their own business. About one-quarter to one-third (27% to 36%) of the clients surveyed believe that the assistance they received had a significant impact (a rating of 4 or 5). Average ratings ranged from 2.5 in terms of encouraging clients to start their own business to 2.9 in terms of further developing their business, management or personal skills. It should be noted that the average ratings are reduced by the fact that each type of impact is not relevant to each client; for example, only a portion of clients are looking for services which would assist them in accessing other sources of funding or further developing their business management skills.

The nature of the impacts varies somewhat between loan and non-loan clients. As illustrated in the following table, non-loan clients are more likely to report that the CFDC assistance had a major impact in enabling them to further develop their skills or encouraging them to start their own business.

Table 6: Loan and non-loan client perceived impacts generated by the CFDC assistance
Impact Loan Clients (N=295) Non-Loan Clients (N=188)
% Reporting
4 or 5
Avg.
Rating
% Reporting
4 or 5
Avg.
Rating
Further develop business, management or personal skills 34% 2.8 39% 3.0
Improve their business practices 29% 2.6 30% 2.7
Increasing access to other programs and services 31% 2.7 25% 2.6
Encouraging them to start their own business 24% 2.4 33% 2.8

Clients reported developing a wide range of skills related to accounting, bookkeeping and finance (identified by 64 clients), marketing and advertising (51), business planning and strategy development (42), general business skills (32), management (17), networking and communications (17), and human resources (9). Improving business practices were reported in the areas of accounting and finance (47), marketing (22), operations (20), customer relations (16), planning (15), human resources (12), reporting and accountability (9), and networking and partnerships (5). Some clients reported gaining access to programs and services including funding or financial programs (identified by 40 clients), business information (34), the business community (16), training and workshops (15), marketing and advertising (10), and other counselling (8). The hands on advice and counselling (61), access to financing (39), and access to information, tools, training and other resources (39) provided by the CFDCs encouraged some clients to start their own business.

Of the 512 clients surveyed, 413 (81%) reported currently operating a business. 

Of the 309 loan clients surveyed, 286 (93%) were operating businesses, 20 businesses had ceased operations (temporarily or permanently including those which had been sold), and three had not started (two are still in the start-up process). Of the 203 non-loan clients 63% were currently operating, 12% were still in the planning stages, 9% had been planning to start a business but will not be proceeding, 7% had ceased operations (temporarily or permanently), and 9% had approached the CFDC for reasons other than starting a business.

In comparison to non-loan clients with active businesses, businesses which have received CFDC loans tend to have been operating for a longer period of time (70% have been operating for more than four years as compared to 46% of non-loan client businesses), generate greater revenues ($945,000 as compared to less than $500,000 for non-loan clients; the median revenues levels are $285,000 and $79,000 respectively), and have more employees (an average of 8.8 employees as compared to 5.5 employees amongst non-loan client businesses).

The CFDC loan clients reported investing an average of $1.50 in funding from other sources for every $1.00 in loans provided by the CFDC. The most common sources included investments by the client or partners, commercial lenders and family and friends.

The surveyed loan clients reported receiving an average of 1.2 loans from the CFDCs, totaling nearly $98,000. The loan client businesses (including those that said no other funds were invested) reported investing an average of $147,000 in funding from other sources, which represents an average of $1.50 in funding from other sources for every $1.00 provided in Community Futures loans. The most common sources of the leveraged funding included the client or partners (80%), commercial lenders (28%), family and friends (12%), private equity investors (7%), and other government sources (5%). The CFDC loans and other funds were most commonly used to start up operations (40%), fund capital or leasehold improvements (35%), expand production (35%), cover working capital or operating costs (32%), fund market development (16%), or fund new product development or commercialization (14%).

The average leverage reported by the loan clients surveyed per loan dollar provided is somewhat lower than the average leverage reported in the CFDC quarterly reports. According to those reports, CFDCs leveraged an average of $2.00 for every $1.00 loaned by CFDCs from 2009–/10 to 2012–13 (over the four year period $330 million in funding was leveraged from other sources by the $165 million in loans provided by the 37 CFDCs).

The revenues of the CFDC loan client businesses that were surveyed have grown at an average rate of about 16% per year since the first important service (usually a loan) was received.

An objective of the survey was to develop an estimate of the impact of CFDCs on client revenues and employment. The evaluation focuses most heavily on loan clients because those are the clients most likely to be impacted by the assistance provided and for whom the impacts are most easily measured and extrapolated. Loan clients were asked a series of questions regarding the status and revenues of their business at the time when the first CFDC service (usually a loan or assistance with business planning, assistance in preparing a business plan, or one-on-one counselling that led to the loan) was provided that significantly impacted the business as well as the size of the business in the most recent year (e.g., revenues, employment, and wages). The average time period that has elapsed since they received that assistance was 4.7 years and the revenues in the year prior to receiving assistance averaged $518,000. A regression analysis indicates that, since that time, the revenues of these businesses have increased at the rate of 16.3% annually. At that average rate of annual increase, the revenues of the average business receiving a CFDC loan would increase from $518,000 in the year prior to receiving the first important service to over $1.1 million five years after receiving the service.

Most loan clients indicated they would not have been able to develop their business to the same degree in the absence of the CFDC loan and other assistance and rated the services as being very important to the development to their business.

Most loan clients believe that the loans and services they received were very important in the establishment or further development of their business. When asked to rate the importance on a scale of 1 to 5, where 1 is not at all important and 5 is very important, the average rating was 4.7. Over three-quarters (79%) of loan clients indicated that it was very important. On average, the businesses estimate that there was only about a 42% chance that they would have developed their business to the extent that they did in the absence of the program. Twenty-seven percent (27%) of the clients felt that there was no chance of developing the business to this extent in the absence of the program whereas 8% felt that the loans and services had no impact on the development of their business.

Most non-loan clients currently operating a business reported growth in sales and believe that the CFDC services were important to them in establishing or developing their business. On average, only about a quarter of non-loan clients surveyed perceived that there is a significant chance that they would have developed their business to the same extent in the absence of the services they received from the CFDC.

On average, the non-loan clients reported that their revenues have increased by an average of 15% per year since the first important service was received. When asked to rate the importance of the services on a scale of 1 to 5, where 1 is not at all important and 5 is very important, the average rating given by the non-loan clients was 3.9; 68% rated the services as important or very important (ratings of 4 and 5).

When asked how likely it is that they would have developed their business to the same extent even in the absence of the program, only about 25% of non-loan clients perceived that there is a 75% or greater probability (of the 127 out of 203 non-loan clients that responded, the average probability was 53%). Of the same 127 non-loan clients that responded, 39% felt that the services had only a small impact or no impact on the eventual development of the business. Although business advice, guidance, training and expertise of staff at the CFDC were very important in helping them focus their resources, develop more efficient practices and connect them to other business, these non-loan clients felt that they would likely have started or developed their business even without the CFDC.

According to key informants, CFDCs are successful in helping businesses to start-up, maintain operations, grow and leverage other services and support.

When asked to rate how successful the CFDCs have been in enhancing the development and survival of businesses in their community or region, CFDC representatives, community partners, and other stakeholders provided average ratings of 4.4, 4.4, and 3.7 respectively, on a scale from 1 to 5, where 1 is not at all successful and 5 is very successful. Key informants emphasized that CFDCs have been particularly successful in helping to maintain jobs and businesses during the recent economic recession (especially in the manufacturing and transportation sectors) and helping businesses to start-up and create jobs by providing training, counselling services and loans that banks are not willing or able to provide due to the level of risk.

B. Strategic Planning, Projects and Partnerships

The CFDCs have played an active role in the development of numerous community strategic plans and implementation of many CED initiatives, most of which would not have gone ahead or have been delayed or reduced in scope or quality in the absence of the CFDC involvement. In turn, these plans and initiatives have brought together many partner organizations, leveraged funding from a wide variety of sources, and enhanced the capacity of communities for socio-economic development.

1. Outputs and Immediate Outcomes

Community Strategic Planning

Most CFDCs reported being active in community strategic planning and almost two-thirds of community partners have worked with a CFDC in developing a community strategic plan.

As part of the evaluation, 61 annual reports submitted by CFDCs from 2009–10 to 2012–13 were reviewed. In these reports, 13 CFDCs reported involvement in the development or update of 34 community strategic plans. However, there is limited reporting available on CFDC involvement in community strategic planning and these figures are not representative of total CFDC involvement.Footnote 37

When asked to rate how active the organization has been in the development of community strategic plans over the past four years, CFDC representatives provided a rating of 3.9, on a scale from 1 to 5, where 1 is not at all and 5 is to a large extent. Of the 37 CFDCs, 23 were rated as particularly active (a rating of 4 or more) in the development of community strategic plans (3 CFDCs reported not being at all active). It was noted that the level of involvement of CFDCs in community planning can vary over time depending on how recently plans were developed and whether other organizations (the local municipality) are filling that role. Sixty-five percent (65%; n=46) of community partners interviewed reported that they had worked with a CFDC in the development of one or more community strategic plans.

The community strategic plans related most commonly to community or regional economic development and other sector specific studies.

According to key informants, CFDCs were most commonly involved in community and economic development strategic plans for local governments such as municipalities, counties and First Nations band councils (identified by 44 key informants). Some key informants (16) also noted that the CFDCs are involved in regional economic plans or partnerships. One key informant noted that the local CFDC participated in community planning activities with 13 different municipalities and 3 counties. As a result, CFDCs assist local governments by engaging the community to assess needs and collecting economic data.

CFDCs also assist with strategic plans focused on specific community organizations such as economic development organizations (e.g., chambers of commerce, workforce development boards, business improvement associations, and sustainable development groups) (12 key informants), specific sectors such as arts and culture (15), tourism (13), downtown revitalization (10) and specific issues such as business retention and expansion (10), labour market development (8), and transportation or communications infrastructure development (8). Examples of other topics addressed by community strategic plans in which CFDCs were involved include the environment, economic leakage, immigrant entrepreneurs, forestry, agriculture, attraction of an educational institution, innovation, disaster relief, business incubation, green technologies, local food, a venture capital forum and angel investor network, and the needs of seniors in the community.

It was also noted that the CFDCs have helped First Nations businesses, bands, and economic development departments on reserves to build their capacity to develop community strategic plans. For example, Tecumseh CFDC helped to build local capacity through training in governance, performance management, board operations, and community economic development.

The CFDCs are most commonly involved in providing funding, input, professional and/or administrative support, and in-kind support. 

Key informants reported that CFDCs played various roles in the development of these plans, noting most commonly that the CFDC provided funding (65% of CFDC representatives and 89% of community partners) and participated in consultations (78% of CFDC representatives and 74% of community partners). Other major roles included providing administrative, in-kind, and other support (e.g., project management, writing, advice, data collection, research, and use of board room or office space for the meetings). Fifty-nine percent of CFDCs (60% in the Eastern Region and 59% in the Western Region) took the lead in the development of one or more plans.

The role of the community partners varies from plan to plan and region to region. The majority of community partners interviewed indicated that they participated in consultations (85%) and took the lead (67%) in the development of community strategic plans. Only 39% indicated that they provided funding for the plan.

CED Projects

The primary objectives of CFDCs, with respect to CED, are to foster economic development, strengthen regional economic development capacity, and support partnerships to undertake CED.

In the annual reports, the CFDCs place a strong emphasis on working with community stakeholders to "act as a catalyst and facilitator for a community identified goal or objective," "enable communities to set their priorities, identify and develop their own capacity and resources and make the best investments," and "increase the intellectual (organizational structure and capacity, planning capability) and human capacity (leadership, skills development, partnerships) within a community." The CFDCs also stress the importance of connecting partners and developing regional strategic alliances (e.g., between local governments).

According to available Annual Reports, 37 CFDCs reported involvement in 2,317 CED initiatives from 2009–10 to 2012–13. CFDCs used CFP-funded staff and in-kind resources to catalyze projects and deliver $8.7 million in funding from sources other than the CFP, and leverage $45.5 million from partners for these CED initiatives.

CFDCs do not have dedicated funding for CED projects and, as such, their contributions come from the CFDC's operating budget or, more often, are accessed from other sources. In 2012–13, less than $200,000 of the 37 CFDC contributions to CED projects ($193,660) came from their operating budgets.Footnote 38 Most of the CFDC contributions consisted of funding accessed by the CFDC from programs such as the Eastern Ontario Development Program (EODP), which is available to each of the 15 CFDCs in eastern Ontario and the Community Adjustment Fund (CAF), which provided a smaller amount of short-term funding for the 22 CFDCs in western Ontario. CFDCs have also implemented projects with other funding from FedDev Ontario (e.g., Economic Development Initiative and Scientists and Engineers in Business program), as well as other federal and provincial government programs (e.g., within the Ontario Ministry of Training, Colleges and Universities), and private organizations.Footnote 39

As shown in the table below, based on activities reported in CFDC annual reports, the majority of CED initiatives focus on economic and business development (1,820 projects with $6.9 million in CFDC contributions), followed by social/cultural (382 projects with $1.1 million in CFDC contributions), and infrastructure (93 projects with $0.8 million in CFDC contributions).Footnote 40

Table 7: Profile of CED initiatives supported by CFDCs, 2009–10 to 2012–13
Type of CED Initiative Number of CED Initiatives CFDC Contribution
(delivery of non-CFP funding)
Partner Contribution Total Value of Initiatives
Economic/Business 1,820 $6,859,905 $27,499,926 $34,359,831
Social/Cultural 382 $1,111,002 $7,663,175 $8,774,177
Infrastructure 93 $775,900 $10,306,284 $11,082,184
Not Defined 22 -- -- --
Total 2,317 $8,746,807 $45,469,385 $54,216,192

CFDCs have supported a variety of CED initiatives during the past four years. The initiatives identified most commonly in the annual reports and key informant interviews relate to arts, culture, and tourism promotion (36 projects noted in annual reports; identified by 37 key informants), entrepreneurial development and capacity building (32 projects; 35 key informants), youth engagement and entrepreneurship training (29 projects; 21 key informants), agriculture and food (16 projects; 33 key informants), labour force development (25 projects; 20 key informants), and economic development events and partnerships (25 projects; 18 key informants). Other examples include downtown revitalization, infrastructure development, business retention and attraction, social enterprises/not-for-profit development, the environment, women in business, innovation, and First Nations capacity building.

Most CFDCs are active in CED projects and have taken the lead in implementing CED initiatives. According to annual reports and key informant interviews, the roles of the CFDCs consist primarily of providing leadership and funding as well as participating in consultations.

When asked how active the CFDC has been in the implementation of CED initiatives over the past four years, CFDC representatives provided a rating of 4.2 on a scale from 1 to 5, where 1 is not at all and 5 is to a large extent. Of the 37 CFDCs, 32 were rated as particularly active (a rating of 4 or more) in the implementation of CED projects. In addition, 85% of the community partners interviewed reported that they had worked with a CFDC in the implementation of CED initiatives.

Most CFDCs (92%) have led one or more CED projects (93% in the Eastern Region and 91% in the Western Region). According to available Annual Reports from 2009–10 to 2012–13, the CFDCs took the lead role in 689 (30%) of the 2,317 CED initiatives in which they were involved.Footnote 41 When asked about the roles of the CFDCs in the CED projects, key informants most commonly identified providing funding (identified by 78% of CFDC representatives and 85% of community partners), participating in consultations (68% of CFDC representatives and 64% of community partners), taking the lead (61% of CFDC representatives and 38% of community partners), and providing administrative support. CFDCs have also supported initiatives by acting as a catalyst to initiate the project before passing the lead to a partner, proposal writing, and planning.

Most community partners indicated that they took the lead (74%) and participated in consultations (85%) in the implementation of one or more CED projects. Representatives explained that, in some cases, they were the co-lead or provided technical leadership and coordination support. Only 51% indicated that they provided funding for the initiative.

Partnerships

In addition to the CFDC, there were an average of over three partners involved in each CED project and an average of over two partners involved in each community planning project. Key informants explained that partnerships were developed primarily with local governments, community organizations, and similar business development organizations.

According to the CFDC Annual Reports, there were 7,520 partners involved in the 2,317 CED projects and 70 partners involved in the 34 community strategic planning activities. Most CED partners were involved in economic/business projects (5,591 partners) and social/cultural projects (1,632 partners).Footnote 42 When asked about the types of organizations with whom partnerships were developed, the CFDC representatives identified local governments including the economic development representatives from municipalities (36 CFDC representatives) and counties (18); regional groups (15) such as economic development committees or agencies; not-for-profit organizations such as chambers of commerce/boards of trade (29), universities and colleges (16), and other not-for-profit organizationsFootnote 43 (36); the private sector including businesses (11) and financial institutions in providing co-financing or promoting small business (6); other federal or provincial government organizations (12) such as the Small Business Enterprise Centres, the Innovation Centres, the Ministry of Agriculture, Farms, and Rural Affairs, National Research Council, and other funders; and First Nations band councils (4).

Community partners attribute the success of CFDCs in partnering with different groups to being well-connected, politically neutral, and long established in the community with a strong reputation. However, the effectiveness of the CFDCs in developing partnerships can vary from community to community.

When asked to rate how successful they believe the CFDC has been in building effective partnerships related to community strategic planning or CED initiatives, CFDC representatives, community partners, and other stakeholders provided average ratings of 4.4, 4.6, and 3.9 respectively, on a scale from 1 to 5, where 1 is not at all successful and 5 is very successful. Community partners attributed the success of CFDCs in developing partnerships to being well-connected (28 community partners) with the different municipalities, industrial sectors, business groups (such as the chambers of commerce), industry associations, other levels of government, First Nations groups, educators and others in the region. As a regionally focused, politically neutral organization with a high profile and good reputation in the community, they are well-positioned to bring different organizations together to work on regional plans or CED initiatives. It was also noted that CFDCs are widely recognized as a key resource with which to partner (21 community partners) as they have a strong willingness to partner, relevant experience, and expertise and can help access to funding or provide administrative support.

A few key informants noted that the effectiveness of the CFDCs in developing partnerships can vary somewhat across communities, depending upon the level of priority placed on CED and strategic planning by the CFDC, the capabilities of other local economic development organizations (and the need for assistance from the CFDC), the willingness and ability of other organizations to get involved in CED and planning, and the relationships between the different organizations (e.g., some work in competition with others).

Some CFDCs partner with other similar service providers through co-location or direct service delivery agreements. These have benefitted CFDCs, partners, and businesses by improving collaboration and expanding the range of services and financing options for businesses.

There is a trend among some CFDCs (e.g., Trenval, Prince Edward, Lennox & Addington, and Valley Heartland) to take partnership to the next level by co-locating with similar service providers and organizations. These CFDCs work with partners such as the provincial government's Small Business Enterprise Centres to provide business counseling to their clients, and with municipal or county economic development groups to plan and implement projects in their region.

Spotlight on Best Practices and Opportunities for Improvement – Partnerships and co-location enhance coordination and access to services and leverage operational costs: A case study of the Trenval Business Development Corporation partnership model illustrates how co-location can benefit the partners through increased communication, reduced duplication of service, more efficient collaboration: "collaborating at the speed of trust" (i.e., since partners know and trust each other which facilitates in-person "hallway" conversations about the support that they could provide for a particular client), enhanced awareness of services among businesses due to increased referrals and joint marketing, and increased understanding and connection to the region and its needs. Businesses benefit from increased access to a broader range of services (e.g., business counselling from the CFDC and the Small Business Enterprise Centre, and mentoring for high tech start-ups from Launch Lab) and financing (e.g., co-financing arrangements from the CFDC, BDC, and CYBF). In addition, the CFDC can benefit by sharing some operating costs with partners (e.g., rent, reception staff, office equipment, meeting space, etc.).

The aggregation of otherwise separate interventions to add value to the system was also identified in the Organisation for Economic Co-operation and Development (OECD) review as a best practice for economic development support and service delivery. The OECD review argues that regional development agencies can play a critical role in aggregating otherwise disparate efforts, overcoming potential coordination failures and information asymmetries. By having a local presence and a flexible structure which allows services and activities to be tailored to local needs and conditions, the CFDCs are well-positioned to facilitate a coordinated and integrated approach to both small business development and community economic development.

2. Intermediate Outcomes

Most key informants rate the CFDCs as successful in strengthening community capacity for socio-economic development, by increasing access to capital, strengthening the entrepreneurial base, and further building the economic development capacity of local organizations. However, more work is required.

When asked to rate how successful they believe the CFDC has been in strengthening the capacity of the community for socio-economic development, CFDC representatives, community partners, and other stakeholders provided average ratings of 3.9, 4.3, and 3.4 respectively, on a scale from 1 to 5 where 1 is not at all successful and 5 is very successful. When asked which CFDC activities have the greatest impact in strengthening the community capacity, key informants most commonly identified the work done to promote small business development (44 key informants) such as increasing access to capital, counselling, training, and other resources. Key informants explained that, by making it easier to start, maintain, and grow businesses, CFDCs help to create and maintain jobs in the community which contributes to socio-economic well-being. A few key informants (3) also explained that, in response to FedDev Ontario's priorities, CFDCs are now placing more emphasis on direct lending and support services for SMEs and less emphasis on other types of CED.

Other key informants highlighted the roles that the CFDC has played in building partnerships, supporting key initiatives, and strengthening the capacity of community organizations (44 key informants). More specifically, the CFDCs have contributed by fostering strategic partnerships, supporting the development of strategic plans and studies, and supporting specific CED initiatives related to key issues (e.g. labour market development, youth engagement, social enterprises, agriculture, tourism, infrastructure, and downtown revitalization).

Those who rated the CFDCs as less successful in strengthening community capacity noted the limited availability of funding for CED initiatives, that CFDCs are placing less of an emphasis on strategic planning and CED projects than in the past, and there is only so much that a CFDC can achieve given the significance of the issues facing some communities and the limited resources of the CFDC.

The case studies illustrated a number of ways in which CFDCs have strengthened the capacity of their communities for socio-economic development, such as enhancing the business skills of youth, improving community infrastructure, increasing awareness and pride in the region, strengthening relationships with partners, building the capacity of the community to engage in sustainable economic development, and strengthening the capacity of social enterprises.

The four case studies conducted as part of this evaluation illustrate some of the different ways in which the CFDCs have strengthened the capacity of their communities for socio-economic development:

3. Incrementality and Attribution

According to key informants, on average, there was a 42% likelihood that community strategic plans would have been developed without assistance from the CFDC.

The perceived likelihood that community strategic plans would have been developed without assistance from the CFDC varied from 45% amongst CFDC representatives to 38% amongst the community partners. Almost three-quarters of CFDC representatives (73%) and community partners (72%) perceived that there was a 50% or less chance that the community strategic plans would have been developed without assistance from the CFDC. Only 10% of CFDC representatives and 7% of community partners perceived that the plans would have definitely gone ahead without assistance from the CFDC.

Those key informants who rated the probability of proceeding without CFDC assistance as low noted that the CFDC played a key role as a catalyst, leader, and first funder (23 key informants), in leveraging additional funding and partnerships (15), in providing critical in-kind support (e.g., space, materials and equipment) and expertise to the plans (15), and serving as a key funder when no other funding source would have been available (15).

A number of key informants (15) noted that the planning may still have gone ahead as planned because the CFDC was involved only as a partner and the local government took the lead. It was noted that local governments with more funding (e.g., in larger communities) and with economic development officers are more likely to be able to proceed without involvement from the CFDCs. Several key informants (14) indicated that, without the involvement of the CFDC, the plans could have proceeded but would have been of reduced quality and scope as a result of less community engagement, fewer partners, and less funding from provincial and federal sources (since the CFDC is often the conduit of such funding). Other ways in which the plans would have been affected include that they would have been developed but not implemented (5), they would have taken a longer time to develop (4), they would be less aligned with economic development plans in neighbouring regions (3), and they would have cost more to develop.

According to key informants, on average, there was a 31% likelihood that CED initiatives would have been implemented without assistance from the CFDC.

The perceived likelihood that CED initiatives would have been implemented without assistance from the CFDC did not vary significantly between CFDC representatives (31%) and community partners (31%). Eighteen percent of CFDC representatives and 25% of community partners indicated that the CED initiatives would not have been implemented without assistance from the CFDC. Key informants who indicated it was unlikely that CED initiatives would have been implemented explained that the CFDC initiated the project (22 key informants), provided critical seed money which leveraged funding from partners (20), or provided necessary project management and administrative support (7).

Those key informants who noted some projects would have been implemented in the absence of the CFDC explained that, in some cases, the CFDC was not the lead, a key source of funding, or the catalyst for that project. Larger projects and projects in larger communities were considered most likely to go ahead without the involvement of the CFDC. Some key informants explained that the projects may have gone ahead but would have been significantly reduced in scope and quality (9) because some activities may not have gone ahead, the projects would have been less organized, and the budget would have been reduced, less successful (4), or conducted over a longer time frame (2) since it would have taken time to find alternative funding and support for the project.

4. Access to CED and Strategic Planning Funding: East vs. West Regional Analysis

CFDCs in the Eastern Region of southern Ontario have received more than four times the level of funding from FedDev Ontario programs (other than the CFP) to support the implementation of CED activities, than those in the Western Region.

As noted earlier, a major factor which influences the level of CFDC activity in CED and community strategic planning is their access to funding from sources other than the CFP. CFDCs do not receive CFP funding dedicated specifically for CED projects but have accessed funding for CED activities through the EODP and, to a lesser extent, the CAF, which are both administered by FedDev Ontario. The EODP, which can be accessed only by CFDCs in the Eastern Region, was established to assist communities in overcoming challenges associated with lower population densities, greater distances from larger economic centres (e.g., from the municipalities that comprise the Greater Toronto Area), the loss of manufacturing jobs, infrastructure needs, and other economic challenges.Footnote 44 While CFDCs in the Western Region do not have access to the EODP, they have received some funding for CED activities from the CAF. As shown in the table below, over the past four years, Eastern CFDCs had received over four times the level of CED funding, per CFDC, from the EODP than those in the Western Region received under CAF ($2.1 million vs. $450,000 per CFDC).

Table 8: funding for CFDCs from the community adjustment fund and the eastern ontario development program, 2009–10 to 2012–13
Number of CFDCs CED Funding Authorized
(EODP/CAF)
CED Funding Expended
(EODP/CAF)
Total For All CFDCs
Total 37 $41,728,522 $41,653,215
East 15 $31,823,195 $31,799,002
West 22 $9,905,327 $9,854,213
Average Per CFDC
Overall - $1,127,798 $1,125,763
East - $2,121,546 $2,119,933
West - $450,242 $447,919

Most CFDCs in both the Eastern Region and Western Region reported being active in community economic development and planning. However, reflecting the differences in the level of EODP/CAF funding accessed, the CFDCs in the Eastern Region reported involvement in a greater number of CED projects.

When asked how active the CFDC has been in the implementation of CED initiatives over the past four years on a scale from 1 to 5, where 1 is not at all and 5 is to a large extent, the average rating was similar for CFDC representatives in the East and those in the West (4.3 and 4.2 respectively). However, according to Annual Reports, CFDCs in the Eastern Region reported involvement in an average of 88 CED projects as compared to 45 CED projects reported by CFDCs in the Western Region, reflecting the differences in the level of EODP/CAF funding accessed. Regional differences in access to funding programs should be considered in assessing the relative performance of individual CFDCs with respect to CED and strategic planning activities. In addition, the need for and impacts of this type of funding should be assessed in evaluations of such programs (e.g., of the EODP and CAF).

According to community partners, community strategic plans in the Eastern Region were much less likely to have been prepared in the absence of assistance from the CFDC than were those in the Western Region (35% vs. 49%). However, there was not much of a difference in the likelihood that CED initiatives would have been implemented without assistance from the CFDC between the Eastern and Western Regions (32% vs. 30%).

C. Ultimate Outcomes

The results of the evaluation indicate that the CFP has generated a significant impact in terms of economic stability, growth and job creation and a positive but lesser impact in terms of supporting diversification and economically sustainable communities as described in the following paragraphs.

The Business Number analysis using Statistics Canada data, combined with the results of the client survey, clearly demonstrate that the CFDC loans and related services have a significant impact in terms of promoting economic growth, job creation and economic diversification amongst the small business sector. Over a five year period, it is projected that the average annual value of loans ($41.34 million) issued by CFDCs in southern Ontario generated an additional $516 million in revenues, 3,865 jobs, and almost $130 million in wages.

The RDAs, including FedDev Ontario, have contracted with Statistics Canada to provide statistical data that compares the performance of businesses in their respective regions that received loans from CFDCs, using CFDC client Business Numbers, with the performance of a comparable group of companies selected on the basis of size, age, sector and region of operation. Three reports have been prepared to date, comparing performance over five year periods ending in 2008, 2009 and 2010. Relative to the comparison group, the CFDC loan clients in southern Ontario have outperformed in terms of business survival rates (across the three rounds of data,Footnote 45 an average of 88% of loan client businesses have survived after five years versus 66% amongst the comparison group) as well as average growth in revenues (14.9% versus 6.9%), employment (10.0% versus 5.9%), and wages (8.7% versus 3.9%). The fact that employment and wages are growing slower than revenues indicates that productivity increases and wage costs decrease as a percent of revenues as business revenues grow.Footnote 46

Across the three rounds, the average size of the CFP-assisted firms included in the analysis was between $1.1 and $1.2 million in revenues; as noted earlier, according to the survey results, the revenues of the average client business receiving a CFDC loan increase from $518,000 in the year prior, to receiving assistance to over $1.1 million five years later, which represents an average annual rate of growth of 16.3%. The average annual growth rate in revenues of the loan client businesses of 14.9%, as per the Business Number analysis, is slightly lower than the rate derived from the client survey which may be attributable to the impact of the economic recovery (the periods covered by the data sets are different) and to revenue growth being somewhat higher amongst larger firms (firms that received larger loans were somewhat over-represented in the survey).

The Business Number analysis provided by Statistics Canada and the results of the loan client survey can be used to project the impacts that are resulting from the loans and related services provided by the 37 CFDCs in southern Ontario over the past four years. More specifically, the analysis uses the results of the survey regarding the average size of the businesses at the time of receiving assistance as well as the current level of revenues and wages paid per employee as of the latest fiscal year (i.e., on average, about five years after receiving the loan and other assistance); Statistics Canada data regarding average growth in revenues, wages and employment over a five year period and average survival rates for both the CFP-assisted firms and the comparison firms; and CFP administrative data regarding the average annual total value of new loans issued over the past four years.

Based on this data, in comparison to the control group, it is projected that for every $1.00 in loans provided by the CFDC, loan clients generate an additional $4.46 in revenues and $1.06 in wages in the fifth year after the loan was received and $12.49 in revenues and $3.13 in wages in the aggregated five year period since the loan was received. Over a five year period, the average annual value of loans ($41.34 million) issued by CFDCs generates an additional $516 million in revenues, 3,865 jobs, and almost $130 million in wages. The value of loans issued per job created is equal to $30,698 in the fifth year after the loans are received and $10,696 in the aggregated five year period.

Table 9: Calculation of economic benefits attributable to the CFDC loans in the fifth year after the loan and in the aggregated five year period
Year Year 0 Year 1 Year 2 Year 3 Year 4 Year 5 Aggregate
Businesses Receiving CFDC Loans
CFDC Per Firm Data Used (Business Number Analysis and CFDC Client Survey)
Revenue Growth -- 14.9% 14.9% 14.9% 14.9% 14.9% --
Employment Growth -- 10.0% 10.0% 10.0% 10.0% 10.0% --
Wage Growth -- 8.7% 8.7% 8.7% 8.7% 8.7% --
Survival 100.0% 97.5% 95.0% 92.6% 90.3% 88.0% --
Average CFDC Loans $97,615 -- -- -- -- -- --
Revenues $518.089 -- -- -- -- -- --
Revenues Per Worker -- -- -- -- -- $107,997 --
Wage Per Worker -- -- -- -- -- $30.400 --
Average Per CFDC Firm That is Active
Revenue $518,089 $595,284 $683,982 $785,895 $902,993 $1,037,539
Employment 6.0 6.6 7.2 7.9 8.7 9.6
Wages $192,450 $209,194 $227,393 $247,177 $268,681 $292,056
Average Per CFDC Firm That Survives (average per active firm multiplied by survival rates)
Revenue $518,089 $580,258 $649,886 $727,870 $815,212 $913,034 $4,204,350
Employment 6.0 6.4 6.9 7.4 7.9 8.5 42.9
Wages $192,450 $203,913 $216,058 $228,927 $242,562 $257,009 $1,340,920
Comparable Firms
Comparable Per Firm Data Used (Business Number Analysis and CFDC Client Survey)
Revenue Growth 6.9% 6.9% 6.9% 6.9% 6.9% --
Employment Growth 5.9% 5.9% 5.9% 5.9% 5.9% --
Wage Growth 3.9% 3.9% 3.9% 3.9% 3.9% --
Survival 100% 92.0% 84.7% 77.9% 71.7% 66.0% --
Average CFDC Loans -- --
Revenues $518.089 --
Revenues Per Worker 6.0 --
Wage Per Worker $192,450 --
Average Per Comparable Firm That is Active
Revenue $518,089 $553,837 $592,052 $632,903 $676,574 $723,257 --
Employment 6.0 6.4 6.7 7.1 7.5 8.0 --
Wage $192,450 $199,956 $207,754 $215,856 $224,275 $233,021
Average Per Comparable Firm That Survives (average per active firm multiplied by survival rates)
Revenue $518,089 $509,672 $501,392 $493,246 $485,233 $477,350 $2,984,982
Employment 6.0 5.8 5.7 5.6 5.4 5.3 33.8
Wages $192,450 $184,010 $175,941 $168,225 $160,848 $153,794 $1,035,268
Attribution (Performance of CFP-Assisted Firms Less the Performance of the Comparable Group)
Per Firm
Revenue $0 $70,586 $148,495 $234,624 $329,979 $435,685 $1,219,368
Employment 0.0 0.5 1.2 1.8 2.5 3.2 9.1
Wages $0 $19,903 $40,117 $60,702 $81,714 $103,215 $305,652
Per Dollar in Loan Funding Provided to the CFDC Loans (Per Firm Divided By Average Loans Received: $97.615)
Revenue $4.46 $12.49
Employment $30,698 $10,696
Wages $1.06 $3.13
Projected Impact (Per Dollar Impacts Multiplied by Average New Loans Issued Annually: $41.34 million)
Revenue ($ millions) $184.5 $516.4
Employment 1,347 3,865
Wages ($ millions) $43.7 $129.4

These figures understate the impact of the CFDC loans and services provided to small businesses to the extent that the impacts reported above continue on into the future and no allowance has been included for the impact of services provided to CFDC clients that do not receive a loan (e.g., received only counselling services or training).

Given that both the size and survival rate of CFP-assisted businesses tend to be much higher than those of the comparable group after five years, there is strong reason to believe that the impacts of the CFDC loans and services will continue on and even increase well beyond the five years included in this analysis.

The figures in the table above do not include an estimate of the impact of services provided to clients who did not receive loans. As noted earlier, the surveys with non-loan clients indicate that the revenues of businesses which are currently operating have increased by 15% per year since the services were received. However, there is no reliable basis on which to extrapolate the results to the total population of non-loan clients given that there is limited financial data available from the survey (many of the non-loan clients are not currently operating businesses or have only recently started operations), there is no Business Number analysis using Statistics Canada data available, there are no standard parameters such as loans on which to base the extrapolation, and it tends to be much more difficult for non-loan clients to attribute their growth to the CFDC services given the softer nature of the non-loan assistance.

The individual CFDCs report data on the number of businesses and jobs created and maintained by the services they provide. While it is true that the services provided to non-loan clients have a significant impact on some businesses, there are clear challenges to using the data reported by the CFDCs as clear evidence of the magnitude of that impact. As noted below, over the four year period covered by the evaluation, the 37 CFDCs reported creating an average of 5,000 jobs per year through the investment fund and 3,749 jobs through business counselling services.

Table 10: Number of Jobs Created and Maintained as Reported Annually by the CFDCs
Year Number of CFDCs Investment Fund Business Counselling
Jobs
Created
Jobs Maintained Total Jobs
Created
Jobs Maintained Total
By Year (Total)
2009–10 37 2,450 3,550 6,000 1,584 3,665 5,249
2010–11 37 1,797 3,074 4,871 813 1,267 2,080
2011–12 37 1,574 2,517 4,091 1,054 1,055 2,109
2012–13 37 1,919 3,167 5,086 2,023 3,534 5,557
Total 37 7,740 12,308 20,048 5,474 9,521 14,995
Average Per Year 1,935 3,077 5,012 1,369 2,380 3,749

The major concerns about using the data reported by the CFDCs are inconsistencies with respect to how the data is collected, analyzed and reported across CFDCs. There are no standard procedures for dealing with attribution; the data is based on expectations rather than actual jobs created or maintained; there are no clearly defined timelines with respect to when these jobs will be created or maintained; and the correlation between the employment data reported in the surveys and the data reported by the CFDCs (as per data included in the population samples) was low. The impact on employment estimated using the survey and Business Number analysis was estimated to be 1,347 jobs (five years after the loan) which is equal to 26% of the 5,012 jobs created and maintained reported by the CFDCs.

The key informant interviews and case studies confirm that CFDCs have been successful in contributing to economic stability, growth, and diversification.

When asked to rate how successful they believe the CFDC has been in contributing to economic stability, growth, and diversification, CFDC representatives, community partners, and other stakeholders provided average ratings of 4.1, 4.3, and 3.4 respectively, on a scale from 1 to 5, where 1 is not at all successful and 5 is very successful.

FedDev Ontario representatives were first asked what they see as the primary intended outcomes of the CFP and then asked to rate how successful the Program has been in meeting those outcomes on the same scale of 1 to 5. The intended outcomes identified most commonly included job creation (10 FedDev Ontario representatives), economic growth and long-term community sustainability (7), and diversified and competitive local economies (4). On average, the FedDev Ontario representatives interviewed rated the CFP as successful in achieving these outcomes (average rating of 3.7).

According to all groups of key informants, the CFP has been particularly successful in facilitating small business development, the creation or maintenance of jobs, economic stability (by expanding the economic base), and diversification. Representatives highlighted examples of significant impacts, such as the impact of a local CFDC on a small community where few businesses had previously been operating, the loans and other support provided for higher risk businesses (e.g., small businesses, start-ups in non-traditional industries), the role that a CFDC has played in facilitating business succession; the role a CFDC played in bringing an important local business out of bankruptcy to the point where it now employs 40 people.

The key informants, particularly those as the community level, also noted the important role that CFDCs play in working with various partner organizations in delivering services, developing strategies and implementing CED initiatives to support economic growth, stability and diversification. CFDCs identify key needs and gaps in the community, work with other organizations to provide or access funding, undertake initiatives to address systemic issues (e.g., attracting businesses, investment and skilled workers), and deliver other programs (e.g., Self-Employment Benefit Program and EODP).

Those who provided lower ratings noted that there are limits to what can be achieved by the CFDCs, given finite operating funds, investment funds, and project resources. For example, while the estimated number of jobs created or maintained by the loans provided annually by the CFDCs (1,347 jobs in the fifth year after the loan and 3,865 jobs in the five year period) is significant, it should be noted that these figures represent only about 0.1% to 0.3% of the total number of people employed in the CFDCs regions. It was also noted by some key informants that the business services have tended to be less successful in supporting innovation, diversifying the economies away from traditional industries, and improving competitiveness. Some representatives, particularly those not based in the communities (such as FedDev Ontario representatives and other stakeholders), noted that the strategic planning and CED activities have tended to have less of an immediate impact because of the longer-term nature of the projects, limited availability of funding (particularly CFDCs in the West that do not have access to EODP funding), and a declining priority placed on these activities by some of the CFDCs.

Spotlight on Best Practices and Opportunities for Improvement – The CFP supports CFDCs in attracting new businesses: The case study of Frontenac Community Futures Development Corporation provides an example of how a CFDC can contribute to the development and attraction of new businesses to the region. The Food and Beverage (FAB) Region Initiative was implemented in partnership between Frontenac CFDC and three counties and it aims to attract small scale artisanal food and beverage businesses to the region. Project activities included developing a branding strategy, website, marketing materials, hiring a public relations expert, undertaking outreach work, and developing two iPad apps (how to start a cheese making business and a craft beer brewery), both of which are available on iTunes. As a result of the initiative, businesses have been attracted to the region, started-up and grown.

Spotlight on Best Practices and Opportunities for Improvement – The CFP supports CFDCs in maintaining and diversifying jobs: The case study of Nottawasaga Futures' Green Economy Centre provides an example of a CFDC's efforts to maintain economic stability and jobs in the region. The Green Economy Centre was developed in response to the recent economic recession to maintain jobs by helping businesses to save on other input costs and to train workers in technical skills that would be required for environmental upgrades. The project was funded primarily by the Ministry of Training, Colleges, and Universities, with in-kind support provided by Nottawasaga Futures. Implementation of the project included developing a website, developing and delivering workshops, offering green business counselling and mentoring, and offering 0% interest green loans. The project exceeded all of its goals and targets. As of March 2013, the project results included 298 jobs created or maintained, 126 green job and training opportunities, 542 participants and 327 businesses reached in all activities, 38 workshops, websites, and videos developed, and 23 business profiles developed.

D. Contributing and Constraining Factors

The results of the evaluation demonstrate that the services meet the needs of the clients and the communities served, which is largely attributable to the design of loans and services, the experienced, knowledgeable and dedicated staff and volunteers, and the strong linkages between the CFDCs and other stakeholders and resources in the community. The CFP and the CFDCs have taken steps to address some of the financial and economic factors that constrain the impact of the CFP by making changes to the lending, services and partnering activities as well as to the program funding model to make more effective use of available resources. The major findings are summarized below.

Most clients are satisfied with services that they have received from Community Futures Development Corporations.

When asked to rate their satisfaction with the assistance received from Community Futures on a scale of 1 to 5, where 1 is not at all satisfied and 5 is very satisfied, loan clients provided an average rating of 4.6 and non-loan clients provided an average rating of 3.9. Nearly 80% of clients were satisfied (rating of 4 or 5), noting that the CFDC staff were professional, supportive, helpful, knowledgeable, respectful and responsive to their needs, loans were not available from other sources, and the services met their needs. Without the assistance received, some clients noted that they would not have been able to start, develop or sustain the business. Those who were less satisfied noted that the services were not very useful to them, they weren't able to access the services they needed (e.g., a loan), the costs (e.g. associated with loans or training) were high, and too much information was provided.

The major factors which contribute to the effectiveness of CFDC activities include the experienced, knowledgeable and dedicated staff and board members, the strong communication with partners, and the reputation of the CFDC in the community.

Key informants, including CFDC representatives and FedDev Ontario representatives, noted that the major factor that contributes to the effectiveness of CFDC activities is the enthusiastic, knowledgeable, and experienced CFDC staff, management, and board members. Representatives explained that staff, management, and board members are well-connected and accountable to the community (since they work and live there). Key informants emphasized that the locally-based decision making model provides CFDCs with flexibility to respond directly to community needs and that the longevity, reputation, independence and perceived neutrality of the CFDCs have contributed to their success. Another key factor central to the success of the CFDCs is the willingness and ability of CFDCs to build bridges and maintain communication with partners.

The internal factor identified most often as constraining the effectiveness of CFDC activities was the level of operational funding available, which limits the ability of CFDCs to deliver in-kind services and support community strategic planning, CED projects, and partnerships.

Some key informants and most CFDC representatives explained that the primary internal factor which constrains the effectiveness of CFDC activities is the level of operational funding available, which limits the number of business services provided as well as involvement in community strategic planning, CED, and partnership activities, and makes it difficult to retain skilled staff. Some CFDCs noted that they are less able to undertake CED and community strategic planning as a result of the budget cuts and the new Performance-Based Funding Model. CFDCs rely more on in-kind support for community plans and projects but this puts added pressure on volunteer board members and staff who already have a full workload. In addition, the inability of CFDCs to provide funding support for projects makes it more difficult to attract and leverage funding from partners.

Other internal factors which were reported to limit CFDC effectiveness included delays and changes to CFP programming as a result of the transition from FedNor to FedDev Ontario, the incentives built into the Performance-Based Funding Model which can make CFDCs more risk averse, the lack of flexibility in loan funds (e.g., some CFDC have excess demand and some have idle funds), the lack of flexibility in the funding agreements (e.g., parameters on using a portion of the loan fund for CED), the lack of clarity from FedDev Ontario with respect to expectations around CED and community strategic planning, and the lack of awareness of the success of the CFP among senior government officials.

External factors which limited the effectiveness of the CFDC activities included the 2008 economic recession, challenges associated with the small size of communities, and a lack of willingness among some governments or organizations to partner with the CFDC.

The external factor most commonly identified by key informants as limiting the effectiveness of CFDC activities was the 2008 economic recession, which caused fluctuations in the Canadian dollar, made businesses less willing to take on risk and expand markets, reduced product demand and tourism from the U.S., and contributed to a decline in employment, outmigration of skilled workers, and increased energy costs and global competition. Business development and expansion is also constrained by challenges associated with the small size of the communities (e.g., a lack of transportation, telecommunications and municipal infrastructure). In some communities, community strategic planning, CED projects, and partnership activities were constrained by unwillingness among some governments and organizations to partner with the CFDC or with each other. Part of the reason is due to isolation and a lack of awareness of the CFDCs and part is due to a lack of interest of some local governments in working with neighbouring regions (e.g., more focus on pursuing growth for their own community instead of seeing the bigger picture and developing common regional goals).

CFDCs have implemented various strategies to mitigate internal and external barriers, such as changes to lending activities, increased partnering, and changes to the delivery model and business services. The CFP has also attempted to make more effective use of available resources by making changes to the funding model.

Eighty percent of CFDC representatives indicated that they introduced new strategies or activities in the past few years to address or mitigate internal or external factors that were slowing the progress made. The primary changes included changes to lending activities (e.g., developing micro-loans and loans for specific groups such as youth), changes to business counselling services (e.g., offering mentorship, training and incubation services for businesses), increased partnering and collaboration, and changes to the delivery model (e.g., restructuring staffing and administration, placing increased emphasis on lending). The changes were reported to have resulted in expanded services, increased business activity, increased jobs in the communities, and other impacts (e.g., increased youth retention, improved infrastructure, and increased tourism to the region). Recognizing that the roles and level of activity of the CFDCs varies across regions, FedDev Ontario has also made changes to the funding model in an attempt to better reflect the performance of each CFDC.

E. Performance Measurement

FedDev Ontario meets the requirements of the Performance Measurement Strategy by collecting and reporting data related to a wide range of indicators. While the existing data is generally considered useful, there are continuing issues with the definition of some indicators, comparability of data, and the resources required to collect and report the data as outlined below.

FedDev Ontario collects extensive information on a quarterly and annual basis from the CFDCs to address Performance Measurement Strategy indicator requirements.

According to the 2012 Implementation Review of the Community Futures Program,Footnote 47 FedDev Ontario collects extensive information to address Performance Measurement Strategy indicator requirements, on a quarterly and/or annual basis from the 37 CFDCs. The CFDCs report on the investment fund, business services, community strategic planning, and CED projects. The quarterly data is used to assess and monitor individual CFDC performance as well as prepare communications documents, regional reports, and national program reports. Business Numbers and postal codes are also collected to support the analysis using Statistics Canada data.

Spotlight on Best Practices and Opportunities for Improvement – Business Number analysis using Statistics Canada data: In response to a need identified in 2009, evaluations of the CFP to improve collection and management performance information in order to ensure that success and cost-effectiveness can be measured with greater accuracy, the regional development agencies (RDAs), including FedDev Ontario, contracted with Statistics Canada to provide statistical data that compares the performance of firms in their respective regions that received loans from CFDCs with the performance of a comparable group of firms. 

To conduct the analysis, a list of CFP-assisted firms was provided to Statistics Canada by each RDA. Each record was linked to Statistics Canada's Business Register using the Business Number and postal codes of CFP-assisted businesses. Statistics Canada also produced a set of tables for a group of firms that did not receive any support under the CFP, which were similar in size (employment level), age, operating in the same industrial sector and geographical locations as CFP-assisted firms. The analysis compared the performance of both groups of firms in terms of growth in sales, employment growth, business survival rate, labour productivity growth and diversification index. Three reports have been prepared to date for southern Ontario, comparing performance over five year periods ending in 2008, 2009 and 2010.

The methodology was identified as a best practice in performance data collection and measurement in terms of accuracy (performance data from Statistics Canada is used), attribution (other factors which impact performance are controlled for using the comparable group), and analysis of impacts (impacts of the CFP services can be isolated by comparing the performance of the two groups).

Most CFDC and FedDev Ontario representatives consider the performance data useful for planning and reporting purposes.

Most CFDC representatives (86%) and FedDev Ontario representatives (92%) interviewed indicated that they use the performance data report by CFDCs, particularly data on loans and loan portfolios, outcomes measures such as jobs created/maintained and businesses created/maintained/expanded, number of services delivered and CED activities. CFDCs most commonly use the performance data to improve internal performance and report results to the CFDC board of directors, FedDev Ontario, and the broader community and stakeholders. FedDev Ontario representatives most commonly reported using the performance data to report to the Minister, determine funding levels, and identify key trends and results. When asked to rate the usefulness of the performance data, CFDC and FedDev Ontario representatives provided average ratings of 4.2 and 3.9 respectively (on a scale where 5 is very useful).

The major concerns with respect to performance measurement are the accuracy and utility of some measures, the consistency in reporting across CFDCs, and the level of effort required.

While the CFP is meeting the data collection requirements of the National Performance Measurement Strategy, there are some issues with respect to performance reporting including:

Spotlight on Best Practices and Opportunities for Improvement – There is an opportunity to streamline reporting requirements and strengthen attribution and reliability of performance indicators. With the reduced core budget and increased reporting requirements, some CFDCs may struggle to strike a balance between reporting and achieving their annual performance levels. Both CFDC and FedDev Ontario representatives recommended that actions should be taken to streamline and reduce duplication in reporting, to develop clear and consistently applied definitions for indicators (particularly with respect to inquiries, consultations, and jobs and businesses created or maintained as a result of business counselling and loan services), to implement a more user-friendly and time saving electronic reporting system, to include more qualitative, contextual data in reporting, and to share information on activities and performance with other CFDCs to facilitate benchmarking.

top of page

VI. Performance – Efficiency and Economy

Demonstration of efficiency and economy is defined under the Treasury Board of Canada Secretariat Directive on the Evaluation Function (2009) as an "assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes."Footnote 48

In this chapter, we examine the relationship between program costs and program outputs (operational efficiency) and outcomes (allocative efficiency). We also look at how program costs are being minimized, while optimizing the achievement of program outcomes (economy). Finally, we assess the management of the loan funds, including the extent to which they have been utilized, grown, and minimized loss.

A. Efficiency

This section examines the efficiency of the CFP as a delivery model as well as the efficiency of its delivery in southern Ontario relative to other regions. To assess the delivery model we use a qualitative/mixed as well as descriptive approach,Footnote 49 relying primarily on evidence in the literature, opinions of key informants, and an assessment of the return on investment (e.g., cost per job created). To assess the efficiency of the delivery of the CFP in southern Ontario, we use a cost-comparative approach, looking at both operational and allocative efficiency.

1. Efficiency of CFP Delivery Model

The CFP service delivery model, which is in place across Canada, is considered efficient and effective. Strengths of the model include local delivery, independence from government, the flexible delivery structure, the integrated services, extensive use of partnerships, the long history of the CFDCs in the communities (which has enabled them to build trust and expertise), and the strong commitment of the CFDC volunteers and staff.

Past evaluations have found the delivery model of the CFP to be a cost-effective approach. Both the 2008 Evaluation of the CFP in Ontario and the 2009 Evaluation of the CFP in Atlantic Canada found that the CFP delivery model is cost-effective because it allows for a grassroots approach where decision making occurs at the local level, it operates at arm’s length from government, and it facilitates partnerships with other local community development players.Footnote 50 This finding was confirmed by an OECD in-depth review which examined 16 development agencies in 13 locations worldwide and defined both the key success factors and the challenges associated with economic development agencies and companies. One of the main observations of the review is that "development agencies perform best within effective local development systems."Footnote 51 The locally-based decision making and flexibility in the delivery model allow development agencies to focus on local challenges, build trust with local businesses and clients, and enhance capacity and co-ordination of the local systems. The importance of the locally focused delivery and flexibility of services was also noted in the interviews with various key informants.

Two-thirds of CFDC representatives and most FedDev Ontario representatives agreed that the CFP is designed appropriately given what it has been attempting to accomplish. They noted that the design of the CFP has been tested and refined through many years of operations and is the most effective model because it is locally run, flexible, and encompasses all forms of support (financial and other services). Similarly, 77% of CFDC representatives, 72% of community partners, 63% of FedDev Ontario representatives, and 50% of other stakeholders agreed that the existing delivery structure is cost-effective. In noting the cost-effectiveness of the delivery structure, the key informants highlighted the third party delivery structure (which allows organizations to be more integrated in the community, build partnerships more easily, and better understand the needs of the local businesses); the commitment and leadership provided by the staff and volunteer-based CFDC boards, which enable CFDCs to continue to provide high quality of services, serve large numbers of clients, and achieve outcomes despite rising costs to CFDCs and cuts in operational funding; the knowledge and expertise of CFDC staff who understand the local economic environment, needs and demands, and have gained the trust of local businesses; the flexibility of the model, which enables services to be tailored to meet the needs of local SMEs whether it is the need for financing, advice or mentorship. When commenting on cost-effectiveness, some community partners highlighted the simplicity of application process and reasonable reporting requirements, the collaborative approach to service delivery, and the range of services that are made available.

The CFP generates a strong return on investment. Based on Business Number analysis using Statistics Canada data, for every dollar contributed, the CFP in southern Ontario generated $43.76 in business revenues and $10.97 in wages, and cost $3,053 per job created in the aggregated five year period since loans were received.

The CFP generates a strong return on investment for the Government of Canada in terms of the grants and contributions costs per job. As described in Chapter III, the CFP grants and contributions expenditures over the four years of the evaluation totaled $47.21 million. Using the Business Number analysis and client survey data in Chapter IV, based on an average annual budget of $11.8 million, for every dollar contributed, the return on investment generated from CFDC services provided to loan clients alone is equal to $15.64 in revenues and $3.70 in wages in the fifth year after the loans are issued and $43.76 in revenues and $10.97 in wages in the aggregated five year period since the loan was received. In addition, the cost per job created is equal to $8,760 in the fifth year after the loans are issued and $3,053 in the aggregated five year period.

The cost per job is in line with other similar programs administered by FedDev Ontario, such as the CME-SMART Program, which assists eligible applicants with funding to carry-out projects that will improve their productivity and competitiveness. According to an Interim Evaluation of FedDev Ontario Programs (October 2012), the FedDev Ontario contribution to the program resulted in an average cost of $7,751 per job created and $3,545 per job maintained ($38.85 million in FedDev Ontario contributions from 2009 to 2012 resulted in 5,012 jobs created and 10,960 jobs maintained).Footnote 52

Furthermore, these figures significantly understate the impact of the CFDCs to the extent that the impacts reported above continue on well into the future and no allowance has been included for the impact of services provided to CFDC clients that do not receive a loan (e.g., received only counselling services or training) or the longer-term impacts of CED initiatives and community strategic planning activities.

2. Efficiency of CFP Delivery in Southern Ontario Compared to Other Regions

The costs of the CFP tend to be higher in southern Ontario than in Western and Atlantic Canada in terms of program costs per CFDC staff member and loan issued. However, southern Ontario CFDCs have lower costs per job created or maintained and leverage more per dollar cost.

Using a cost-comparative approach to evaluate the allocative and operational efficiency of the CFP in southern Ontario, the program costs were compared across regions using budget data reported in the National Performance Measurement Strategy and activity and outcome data reported in the annual reports of the Community Futures Network of Canada (the annual reports did not include data for Quebec).Footnote 53

In terms of operational efficiency, the results indicate that the program grants and contributions costs per CFDC and CFDC staff member tend to be higher in southern Ontario than Western and Atlantic Canada. The findings are mixed with respect to allocative efficiency. The costs per loan issued and dollar in loans provided tend to be higher in southern Ontario. In contrast, southern Ontario tends to have lower costs per reported job created or maintained and leverage more per dollar cost compared to other regions.

Table 11: Cost per outputs/outcomes across RDAS for 2010–11 and 2011–12
2010–11 2011–12
WD IC FedDev
Ontario
Atlantic WD IC FedDev
Ontario
Atlantic
Program Costs
G&C Costs Footnote 54
(in thousands)
$22,670 $8,400 $12,540 $12,640 $22,670 $8,400 $12,540 $12,640
Program Outputs
# of Community Futures/CFDCs 90 24 37 41 90 24 37 41
Cost per CFDCs $251,889 $350,000 $338,919 $308,293 $251,889 $350,000 $338,919 $308,293
Estimated
# of Staff
562 145 224 251 525 145 224 242
Cost per CFDC Staff $40,338 $57,931 $55,982 $50,359 $43,181 $57,931 $55,982 $52,231
Program Outcomes
# of Loans 1,474 393 704 1,753 1,443 387 557 1480
Cost per loan issued $15,380 $21,374 $17,813 $7,210 $15,710 $21,705 $22,513 $8,541
Value of Loans
(in thousands)
$71,477 $22,938 $46,261 $61,473 $67,263 $17,691 $28,840 $57,160
Loaned per
$ cost
$3.15 $2.73 $3.69 $4.86 $2.97 $2.11 $2.30 $4.52
Amount Leveraged
(in thousands) Footnote 55
$80,265 $82,080 $101,001 $60,898 $82,463 $31,373 $72,759 $39,765
Leveraged per
$ cost
$3.54 $9.77 $8.05 $4.82 $3.64 $3.73 $5.80 $3.15
Jobs Created & Maintained Footnote 56 7,062 2,314 6,951 4,806 5,515 1,510 4,091 3,562
Cost per Job Created or Maintained $3,210 $3,630 $1,804 $2,630 $4,111 $5,563 $3,065 $3,549

Source: Community Futures Network of Canada Annual Report 2011–2012. FedDev Ontario and IC data come from the CFP Quarterly Report Data.

Major factors that contribute to differences across regions may include differences in the types of services delivered by CFDCs (e.g., Community Futures offices in Atlantic Canada only provide loans and business counselling),Footnote 57 the extent to which Community Futures offices have leveraged operational funding with funding from other sources (e.g., other federal and provincial programs), and environmental factors such as differences in the size of geographic regions, population density, lending environment, economic conditions, and costs of living. In addition, indicators are likely impacted by differences in how performance data is collected and reported across regions. Furthermore, the comparisons should consider whether higher costs per output or outcome are indicative of efficiencies. For example, the CFP in southern Ontario is delivering services with fewer staff than other regions, which could indicate that it is optimizing the use of CFDC staff and volunteer resources.

A review of FedDev Ontario program costs per output illustrates how apparent efficiency can vary widely across fiscal years in response to changes in program budgets, activity levels, and economic conditions.

As illustrated below, the costs of the program per outcome was the lowest in 2009–10 which had a lower program budget and followed the economic downturn, almost doubled in the next year, and continued to increase the following year as the economy recovered. Costs per outcome decreased again in 2012–13 as the program budget was reduced and outputs rose.

Table 12: feddev ontario CFP cost per output/outcome 2009–10 to 2012–13
Years 2009–10 2010–11 2011–12 2012–13
G&C Costs (in $million) $10.8 $12.5 $12.5 $11.3
# of new loans issued 813 704 557 705
Value of new loans (in $million)  $48.0 $46.3 $28.8 $42.2
Loan value per dollar of costs $4.44 $3.70 $2.30 $3.73
Businesses assisted (counseling) 3,097 1,560  1,519 2,936
Number of business assisted (loans) 808 708 556 689
Total number assisted 3,905 2,268 2,075 3,625
Cost per business assisted $2,766 $5,511 $6,024 $3,117
Jobs created (service and loans) 4,034 2,610 2,628 3,942
Jobs maintained (services and loans) 7,215 4,341 3,572 6,701
Jobs created and maintained 11,249 6,951 6,200 10,643
Cost per job created/maintained $960 $1,798 $2,016 $1,062

According to key informants, activity levels are strongly influenced by various external factors such as economic conditions; it was noted, for example, that CFDC lending tends to be cyclical and can increase during economic downturns as bank lending tightens.

There are significant variations in costs per output and outcome across the 37 CFDCs, given that each CFDC received the same base funding while the level of reported activities and outcomes varied widely.

An analysis of individual CFDC Contribution Agreements and financial data breakdown for 2012–13 shows that 30 organizations received $285,000 in base funding, six organizations received an additional $20,000 under official languages funding, and two CFDCs received satellite office funding (one received $50,000 and one received $25,000). As illustrated in the following table, almost three-quarters of the funding is used for staff wages and benefits (72% of the total budget).Footnote 58

Table 13: CFDC operational funding Analysis, 2012–13
Types of Expenditures Number of CFDCs Average per CFDC Total % of total
Wages/Salaries 37 $179,541 $6,643,002 61.9%
Benefits 37  $30,100 $1,113,574 10.4%
Overhead 37 $69,258 $2,562,528 23.9%
CED 20 $9,683 $193,660 1.8%
Capital Cost 14 $2,303 $32,236 0.3%
Official Languages 6 $20,000 $120,000 1.1%
Satellite Office 2 $37,500 $75,000 0.7%
Total Expenditures $10,740,000 100%

Average wages for CFP staff of the 37 CFDCs totaled $6.6 million in 2012–13. The most common positions were executive director or general manager, investment fund or business services officer, office administration and CED coordinator. The average salary per staff member was $46,000, including $75,000 for the executive director and $36,000 for the remaining positions.

Overhead includes various fixed costs such as renting office space. It was found during case studies (see the case study of Trenval Business Development Corporation) that some CFDCs have adopted strategies to, in effect, reduce fixed costs such as subletting space to other organizations (e.g., BDC, Small Business Enterprise Centres, Regional Innovation Centres, etc.) as well as by delivering other programs (e.g., the Self-Employment Benefit Program) for which the CFDC receives an administrative fee that can go towards their overhead expenses. This allows CFDCs to use a larger portion of CFP funding for staff or funding CED projects. On average, in 2012–13, the CFDCs spent only 1.8% of their funding on CED projects (excluding staffing and other in-kind support).

Over the four years of the evaluation, CFDCs in the Western Region of southern Ontario were more efficient on average, than CFDCs in the Eastern Region with respect to annual CFP contribution costs per staff ($71,854 vs. $82,801). However, Eastern CFDCs were more efficient compared to Western CFDCs with respect to cost per loan issued ($11,348 vs. $21,741), and dollars loaned ($4.71 vs. $3.10) and leveraged ($8.80 vs. $6.61) per dollar cost as a result of issuing a greater number and value of loans per CFDC than Western CFDCs.

It was found that costs per output or outcome varied widely across CFDCs. From 2009–10 to 2012–13, for example, the average CFDC cost per business assisted ranged from $735 to $23,173 ($3,709 average for all CFDCs), the average cost per reported job created/maintained ranged from $369 to $5,643 ($1,257 average), and the average dollar loaned per dollar cost varied from $1.23 to $12.16 ($3.75 average). Part of the variation in costs per output and outcome may be due to inefficiencies. However, part of the variation may indicate that CFDCs are responding well to the needs in their region. For example differences in the characteristics and needs of the regions served (e.g., geographic size, population density, economic conditions, and other services available) may cause CFDCs to place a different emphasis on various services (e.g., business loans compared to business information and counselling services).

Four different Community Futures associations deliver services on behalf of CFDCs. While the CFDCs are generally satisfied with the services provided, no clear performance indicators are in place to assess the effectiveness, efficiency or economy of those services.

In addition to the national Community Futures Network of Canada, there are three CFDC associations based in Ontario including the Ontario Association of CFDCs (OACFDC), the Western Ontario CFDC Association (WOCFDCA), and the Eastern Ontario CFDC Network Inc. (EOCFDC Network). FedDev Ontario currently provides operational funding to three of the associations and the funding allocation is determined by the number of CFDCs they serve. Each association receives $5,000 per CFDC (i.e., $185,000 is allocated to the OACFDC, $75,000 is allocated to the EOCFDC Network, and $110,000 is allocated to the WOCFDCA). The associations provide various types of support to member CFDCs including disseminating information (e.g., on best practices), organizing meetings and conferences for CFDCs, offering training to CFDC staff and board members, advocating for CFDCs, and delivering programs which benefit multiple regions (e.g., SOFII) as well as offering cost-savings through group purchasing (e.g., insurance, software, and equipment).

CFDC representatives found the services provided by the Ontario associations to be useful to them (ratings ranged from 3.5 for the OACFDC to 3.7 for the WOCFDCA and 4.0 for the EOCFDC Network where 5 is very useful), noting that they share information about best practices, administrative challenges and tools, offer cost saving opportunities through group purchasing,Footnote 59 and delivered some programming. The CFDCs tended to have less interaction with the Community Futures Network of Canada and therefore perceive the organization to be less useful to them (average rating of 3.1). CFDC representatives recommended that the national and provincial associations increase their level of communication and responsiveness to members, and that that the level of funding (or mechanism through which associations are funded) should be reviewed.Footnote 60 Some key informants questioned the need to fund multiple CFDC associations and networks.

FedDev Ontario representatives rate the usefulness of the services and functions of the Community Futures associations as a group at 2.9. Several FedDev Ontario representatives recommended that they should eliminate FedDev Ontario funding to the organizations, distribute the budget amongst CFDCs, and let CFDCs decide if they want to purchase memberships or pay for services (i.e., make network membership an eligible expense). Representatives also noted that there should be more coordination between the associations and that associations should facilitate additional training for CFDC staff and board members.

Spotlight on Best Practices and Opportunities for Improvement – The relative roles, mandates and performance measures of Community Futures associations should be better defined. There is no consensus regarding how the associations should be structured and funded. However, there was recognition amongst FedDev Ontario and CFDC representatives that it is unusual for there to be four associations in place and that associations can play an important role in areas such as professional development, identification and communication of best practices, promoting coordination and collaboration across CFDCs and with other organizations, and other support services.

Associations could also play a key role in further enhancing marketing and branding of the CFP. Key informants noted that further promotion of the CFDCs would the increase the impact of the CFP by expanding use of services and investment funds. Seven percent of the clients recommended that the CFDCs should do more to market and increase awareness of their services. While marketing can also be done locally and provincially, most CFDCs (74%), stakeholders (67%) and approximately one-half of FedDev Ontario representatives believe that the program should develop a national brand for the CFP from coast-to-coast. A strong national brand would increase awareness of services, provide greater consistency and continuity in marketing, and raise the public and political profile of the CFP across the country while preserving the ability of Community Futures offices to tailor their services to meet local needs.

B. Economy

According to the Treasury Board of Canada Secretariat, economy is said to have been achieved when the cost of resources used approximates the minimum amount of resources needed to achieve expected outcomes. In this case, minimum does not refer to an absolute minimum. Rather, it refers to an optimized and contextualized minimum. This optimized and contextualized minimum is determined by analyzing the degree to which input costs were minimized based on the program's context, input characteristics (such as their quality, quantity, timeliness and appropriateness), and the impact on outcomes achievement.Footnote 61

In this section we assess the extent to which program costs are being minimized, while optimizing the achievement of program outcomes.

In response to CFP budget reductions and a desire to better link results to contribution funding, FedDev Ontario has been working to optimize program outcomes, which has culminated in the recent introduction of the Performance Based Funding Model.

Since FedDev Ontario took over management of the CFP in southern Ontario in 2009, a number of activities have been undertaken in order to optimize program contribution costs.

In August 2010, a request was sent by FedDev Ontario to all 37 CFDCs seeking their assistance in the completion of an organizational review of their CFP-related activities, particularly, information on regional economic challenges, governance, operations and loan portfolios. The CFDC Information Kit that resulted from the review found that there were a number of differences between CFDCs with respect to economic conditions, governance practices and training, and levels of demand for services. The findings led to a number of recommendations including to provide support to develop and deliver governance best practices, to develop a performance based funding model, and to develop investment fund mechanisms such as shared-risk loan pooling. Responding to these recommendations, an initial Performance Based Funding Model was developed in 2011. The model incorporated minimum standards and best practices standards, as well as a performance contribution based on CFDC performance against Key Performance Indicators (KPIs), where targets are set by individual CFDCs.Footnote 62

In 2012, the government-wide budget for delivery of the CFP was reduced on an ongoing basis. The CFP delivery in southern Ontario budget was reduced by 10% ($1.254 million). To achieve savings and continue to optimize the program, FedDev Ontario, in consultation with the CFDCs, moved forward with full implementation of the Performance Based Funding Model (also referred to as the CFP Transformation). Between May and September 2012, a series of engagement sessions were held between FedDev Ontario representatives, CFDCs, associations and other key stakeholders. The sessions were intended to receive input on the funding model as well as determine the indicators and definitions that could be used to measure outputs and outcomes. In November 2012, the implementation of the Performance Based Funding Model was approved by FedDev Ontario as well as other program enhancements described below. A detailed description of the Performance Based Funding Model is provided in Appendix VI.

Under the Performance Based Funding Model, each CFDC will receive $150,000 as an annual base contribution (CFDCs will receive a base of $225,000 in 2013–14 only, as a they transition to the full funding model in 2014–15). Additional funding for each CFDC will be determined based on:

Changes to the funding model are being introduced concurrently with several other program enhancements including increasing CFP loan limits from $150,000 to $250,000 on an exceptional basis; providing a CFP capitalization and capacity enhancement budget to respond to systematic or delivery needs and to fund such options as regional loans pools or strategic capitalizations ($200,000 is budgeted for 2013–14 and $500,000 is budgeted for 2014–15); supporting CFP equity investment opportunities in SMEs by addressing the awareness and training gap at the CFDC level (FedDev Ontario will work with CFDCs and associations to facilitate training and awareness); enhancing CFP collaboration by implementing opportunities for enhanced collaboration within and outside the CFDC network (e.g., pilot projects to align CFDCs and provincially funded Small Business Enterprise Centres); and enhancing the CFP Investment Fund Transfer Policy to provide CFDCs with greater flexibility to address operational challenges (CFDCs have the ability to access 50% of their Investment Fund’s 3-year average net income up to a maximum of $100,000 before a corresponding reduction in base funding, to be used towards CFP eligible costs).Footnote 63

Although it is too early to assess the impact of these activities on the program’s economy, and it is out of the scope for this evaluation, there is some anecdotal evidence that the funding model has resulted in improved optimization of resources, "through the new CFDC reporting requirements the efforts of individual CFDCs to increase both the number and value of loans has been observed, and overall the level of uncommitted funds within CFDC investment fund portfolios is starting to decline as a result of increased loans activity."Footnote 64

While there appears to be sufficient resources to support CFP operations and maintenance costs, the level of resources to support grants and contributions was identified as a major constraint to the achievement of program outcomes, and there are concerns the new funding model may discourage CFDCs from addressing CED and community strategic planning needs.

There appears to be sufficient resources to support CFP operations and maintenance (O&M) costs. As described in Chapter III, the CFP-related operations and maintenance expenditures over the four years of the evaluation totaled approximately $3.08 million, with annual expenditures of about $770,000 per year. In addition, 10.45 FedDev Ontario FTEs are dedicated to administering the CFP. The annual CFP O&M expenditures per CFDC in southern Ontario is similar to most other regions (e.g., $20,811 in southern Ontario vs. $21,700 in Western Canada, $22,125 in Northern Ontario, and $16,512 in Atlantic Canada in 2010–11 and 2011–12). Also, most FedDev Ontario representatives agreed that the resources, particularly in terms of O&M, are adequate to support effective implementation and achievement of expected outcomes.

In contrast, the level of grants and contributions resources for the CFP may be constraining the achievement of outcomes. Only 13% of CFDC representatives agreed that the program resources are adequate. The perceptions were similar in both the Eastern and Western Regions (15% of CFDC representatives in the Eastern Region and 11% of representatives in the West agreed resources are adequate). Of the two-thirds of CFCDs representatives who somewhat or strongly disagreed, most indicated that the reduced funding is not adequate to cover the rising cost of CFDC operations and increasing requirements for reporting. Reductions in funding, combined with uncertainly regarding the level of future funding, may make it more difficult for some CFDCs to attract and retain qualified staff, deliver a full range of services, deal with fluctuations in demand, plan future programming, and achieve expected outcomes. Some representatives expressed concern that unpredictable circumstances (i.e., the closing of a local factory or plant) and other events beyond their control could significantly impact their performance and therefore level of funding. In addition, some FedDev Ontario representatives argued that more resources are needed for capacity building (training, marketing partnership and collaboration), while a few FedDev Ontario representatives noted that additional resources for CFDC operations and investment funds are needed to support organizations in achieving outcomes.

There is also some concern that the new Performance Based Funding Model may discourage certain activities (e.g., supporting community strategic planning or CED projects) that are needed in the community or affect how CFDCs track and report on their activities, partnerships and outcomes. Under the Performance Based Funding Model, only 10% of the outcome-based funding and only seven out of the 20 KPI indicators pertain to CED and community strategic planning.Footnote 65 Some CFDC representatives indicated that the reduced funding may mean that they will no longer be able to have much involvement in community planning or CED, work in partnership with community organizations, or provide the same level of services to clients. As a result, some CFDC and FedDev Ontario representatives recommended that the impacts of the new system be carefully monitored during the first few years of implementation.

Spotlight on Best Practices and Opportunities for Improvement – The impacts of the Performance Based Funding Model should be closely monitored. The new funding model may impact the ability of some CFDCs to meet the demand for services in their communities, influence the types of activities undertaken in a way that is not consistent with local needs (e.g., placing more of an emphasis on loan funding versus CED activities or counselling), and influence how activities and outcomes are tracked and reported. As a result, FedDev Ontario should closely monitor implementation of the Performance Based Funding Model and assist CFDCs, where needed, in developing effective strategies to deliver the same level and quality of services with reduced budgets. There may be a need to refine the funding formula over time to ensure that it promotes cost-effectiveness, needs-driven services and activities, appropriate risk-taking, and greater self-sufficiency.

In addition, aspects of the CFP governance structure constrain the achievement of outputs and outcomes (e.g., frequent changes and duplication in reporting requirements and a lack of guidance and clarity with respect to program objectives).

FedDev Ontario and CFDC representatives differed somewhat in their opinions as to the economy of the governance structure. Only 27% of FedDev Ontario representatives agreed that the governance structure is clearly defined and economical (most did not express an opinion) while 63% of CFDC representatives agreed. Part of the variance in opinions pertains to which aspects of the governance structure key informants were assessing (i.e., governance of at the FedDev Ontario level or CFDC level). Governance at the CFDC level was largely seen as a strength of the program, as noted earlier in this chapter. CFDC representatives perceive that the strengths of the governance structure include strong community representation on the board, a high level of accountability, well-established management processes and systems, clearly defined goals and policies, and a long track record of success.

Any concerns about the governance structure generally focused on the relationship between FedDev Ontario and the CFDCs. CFDC representatives noted some inconsistencies in communications with FedDev Ontario, a lack of clarity in decision processes (project officers do not have authority to make decisions), and that the reporting requirements are onerous and duplicative. About one-third of CFDC representatives reported that changes in reporting requirements and structures have put an additional burden on already overbooked and scarce staff time, and there has not been sufficient guidance and training on the requirements. A few representatives noted that inflexible reporting timeliness and delays in administration of contracts and auditing for claims has restricted efficiencies in their operations. In addition, a few FedDev Ontario representatives suggested that the governance structure involves too many layers of management (e.g., FedDev Ontario does not need to closely manage CFDCs as they are independent, small, non-government organizations well managed by their own board of directors), the role and mandate of various CFDC associations is not very clear, and further efficiencies could be achieved by reducing the number of CFDCs.

Spotlight on Best Practices and Opportunities for Improvement – Communication and relationships between CFDC representatives and FedDev Ontario representatives should be strengthened. Several CFDC representatives recommended that FedDev Ontario staff streamline their management of the CFP and strengthen their relationships with individual CFDCs. It was suggested that project officers could have more frequent face-to-face meetings with CFDCs or sit-in on major project committee meetings so that they are more aware of local community needs and challenges. It was also recommended that FedDev Ontario more clearly communicate expectations of CFDCs with respect to CFP objectives, particularly in areas that receive no additional funding such as CED and community strategic planning. Finally, CFDCs noted that they would like to have their relationship with FedDev Ontario more clearly defined and to be engaged more as a partner in supporting economic and business development rather than just a funding recipient.

C. Management of the Loan Funds

On average, the CFDC investment funds are well utilized. Over the four year period, an average 71% of the investment funds were in active loans. However, there may be opportunities to further optimize use of loan funds in some CFDC regions.

From 2009–10 to 2012–13, the CFDCs managed an average of 2,504 loans valued at $134.4 million annually which represents 71% of the average total investment portfolio of $189.8 million. The percentage of the investment funds in active loans remained relatively stable, ranging from 67% to 74% across the four years.

Table 14: Percentage of the Investment FundS in Active Loans
Year Investment Portfolio Activity
Number of Active Loans Value of Active Loans Value of Investment Portfolio % of Investment Fund Active
2009–10 2,349 $123,774,918 $170,319,516 73%
2010–11 2,561 $138,930,723 $187,185,331 74%
2011–12 2,485 $134,431,546 $199,361,811 67%
2012–13 2,619 $140,410,002 $202,337,966 69%
4-Year Average 2,504 $134,386,797 $189,801,156 71%

The range across individual CFDCs varied to a greater extent. For example, in 2012–13, 11 CFDCs had between 50% to 60% of their investment funds in active loans while five CFDCs had between 85% to 95% of their investment funds in active loans. For all four years, on average, CFDCs in both the Western and Eastern Regions had the same percentage of their investment funds in active loans (71% for both regions).

This level of activity over the four years is similar to that reported in the previous evaluation for Ontario overall (e.g., utilization rates of 72% in 2004–05 and 69% in 2007–08).Footnote 66 The previous evaluation also referenced a study, commissioned by FedNor in 2003, which inferred that CFDCs should maintain approximately 30% of their lending activity in cash reserves. However, the report went on to conclude that a more appropriate minimum level of cash reserves for CFDCs would be 50% of the CFDC’s average lending activity over the past three years.Footnote 67 Utilization of the loan funds appears to be similar to CFDCs in other regions. The 2009 evaluation of the CFP in Atlantic Canada reported the percent of CFDC investment funds in active loans averaged 78% from 2003 to 2008.Footnote 68

According to most FedDev Ontario representatives, the management of the funds is generally appropriate although each CFDC manages their investment fund differently. CFDC representatives reported that effective structures are in place to manage the loan portfolios (e.g., trained staff and committee members supported by comprehensive protocols).

As noted earlier, recent changes to the program and funding model are attempting to address potential inefficiencies and to facilitate increased lending (e.g., introduction of the Performance Based Funding Model, establishment of the CFP capitalization and capacity enhancement budget, which can be used to fund options such as regional loan pools or strategic capitalizations). However, some CFDC representatives noted that it will be difficult for them to increase their loan activity because most of the funds are already utilized, other sources of capital are available locally, or they do not have sufficient operating funds available to market their loans and other services. There may be further areas to improve efficiency, particularly with respect to facilitating loan fund transfers between areas where there is less demand to areas where the loan fund struggles to meet demand.

Spotlight on Best Practices and Opportunities for Improvement – There are opportunities to improve the efficiency of CFDCs in regions where there is less demand for services (e.g., merge functions of CFDCs with less demand for services or co-locate with similar service providers). Some of the suggestions made by CFDC and FedDev Ontario representatives were to share staff and other resources, merge/amalgamate smaller CFDCs or at least certain functions of those CFDCs (e.g., management of the investment funds), broaden the revenue base or reduce costs by accessing provincial or other federal government funding for related services, and co-locate or co-deliver services with other programs or organizations.

The total value of the investment funds grew by 7% (approximately $12 million) from 2009/10 to 2012/13, as a result of positive returns earned on the loans. Recapitalizations through CAF contributed to an additional increase of $29 million.

The total dollar value of the investment funds for all 37 CFDCs grew from $161 million at the beginning of the period covered by this evaluation to over $202 million as of the end of 2012/13.

Table 15: Investment portfolio growth, 2009–10 to 2012–13
Investment Fund Fiscal Year Beginning April 1, Four Year Total
2009 2010 2011 2012
Beginning Balance $161.2 $170.3 $187.2 $199.4 $161.2
Recapitalization $14.3 $14.8 -- -- $29.1
Retained Earning ($5.2) $2.1 $12.2 $2.9 $12.0
Ending of Year Balance $170.3 $187.2 $199.4 $202.3 $202.3
Retained Earnings/Beginning Balance
% Growth −3.2% 1.2% 6.5% 1.5% 7.4%

Of this growth in value, $12 million has been generated from returns on active loans while $29 million consists of additional contributions made into the investment funds by FedDev Ontario. The new capitalization funding provided by FedDev Ontario brought the total investment fund contributions made by the federal government to over $120 million. The amount of federal contributions per CFDC varies from less than $2 million (7 CFDCs) to $4 to $7 million (8 CFDCs). Of the 37 CFDCs, five currently have an investment fund balance which is less than the amount contributed by the federal government over the years while 32 have generated at least some return on the loan funds.

The CFP lending limit of $150,000 and $250,000, on an exceptional basis, is perceived to be appropriate, given the CFP’s focus on small and start-up businesses.

Most key informants perceived that the lending limit of $150,000 and $250,000, on an exceptional basis, is appropriate. About half of FedDev Ontario representatives and over two thirds of CFDC representatives agreed that the current loan limit is appropriate given that the CFP has a focus on smaller, start-up businesses (e.g., whereas BDC typically provides larger loans), and that, where warranted, CFDCs can partner with other CFDCs to effectively raise the loan limit. Some CFDCs warned that too many higher value loans could represent a major risk for a small overall portfolio. However, about one-third of CFDC representatives recommended that increasing the normal loan limit to $250,000 would be helpful (recognizing that most loans would still be for far less), particularly when trying to increase performance of loans against indicators in outcome based funding (e.g., larger loans can potentially create more jobs).

Based on available information, the low loan loss rate of the 37 CFDCs has averaged about 4.5%. While most key informants consider the loan loss rates to be appropriate, there is some concern that the new funding model may cause CFDCs to become more risk averse.

At least one investment report was obtained for each of the 37 CFDCs. The investment reports generally show the loan loss provisions and loans receivable of the CFDC for each of the previous five years. Taken together, the reports summarize the loan loss rates for a sample of 200 instances of CFDCs and fiscal years between 2004–05 and 2011–12. On average, the loan loss rates (aggregate loan loss provisions divided by the total value of the loans receivable) have averaged 4.5% with the annual fiscal year average (based on incomplete data) ranging from 3.3% to 4.7% for the first three years covered by this evaluation.

Table 16: CFDC Loan Loss Rates, 2004–05 to 2011–12
2004–05 2005–06 2006–07 2007–08 2008–09 2009–10 2010–11 2011–12 7 Year Average
Loan Loss Rate 6.1% 2.0% 5.0% 4.7% 5.8% 4.7% 3.8% 3.3% 4.5%
# CFDCs 9 16 25 32 34 34 35 15 37

CFDCs are mandated to provide loans to higher risk clients (e.g., start-up businesses, smaller businesses, businesses in riskier sectors). So a higher loss rate relative to private financial institutions should be expected. Financial institutions such as banks and credit unions typically have loan loss rates under 1%, and the BDC loan loss rate is typically around 1%.Footnote 69 However, the CFDC loan loss rate should not be so high that the loan funds do not continue to generate a sustainable return overall.

Most FedDev Ontario and CFDC representatives agreed that the average loan loss rates are acceptable (73% of FedDev and 83% of CFDC representatives agreed or strongly agreed). Some FedDev Ontario representatives noted that the rate is, for at least some CFDCs, too low given that that the purpose is to fill a niche and support riskier and more innovative businesses. Representatives of CFDCs whose loss rate is below 4% were more likely to suggest that their loss rate is too low while those whose rate was above 4% were more likely to suggest a high rate was warranted given that their objective is to support higher risk clients. Representatives from both CFDCs and FedDev Ontario indicated that there has never been guidance or consensus regarding what constitutes an acceptable loss rate.

Some representatives are concerned that the loan loss rates are already too low and the new funding model may cause some CFDCs to become even more risk averse, given that the model rewards the loans which lead to the creation/maintenance of businesses and jobs (further driving down the loan loss rate). It was also noted that low loss rates do not necessarily indicate that a CFDC is not lending to "high risk" clients as perceived by commercial and other lenders (e.g., start-ups); key informants explained that CFDCs mitigate loan risk by assessing loan applicants and supporting loan clients in ways that are beyond what a typical commercial bank would do. CFDCs consider aspects such as personal character in loan application reviews, and provide loan clients with business counselling and support to help clients to succeed. It was noted that CFDCs should continue to seek these types of clients (e.g., newer or more innovative businesses) in order to fill financing gaps.

top of page

VII. Conclusions and recommendations

This chapter summarizes the major conclusions and recommendations arising from the evaluation.

A. conclusions

The major conclusions arising from the evaluation of the Community Futures Program in southern Ontario are as follows:

Relevance

Continued Need for the Program

1. There is a strong need for all four lines of business of the CFP in southern Ontario (business financing, business information and services, and support for community strategic planning and community economic development projects), given the significant economic challenges faced by smaller communities, the need for support targeted specifically at SMEs, and the important role that CFDCs play in supporting community economic development.

There is a strong need for the CFP in southern Ontario. Smaller communities face significant economic challenges associated with a low and often declining population base; more restricted access to workers, particularly skilled workers; and a heavy reliance on natural resource sectors which makes the economy more vulnerable to economic swings. Most key informants reported a strong need for programming designed to help communities address these issues, including both assistance targeted at SMEs and broader systemic support designed to promote community economic development.

There is a continued need to provide business financing and business information and services for SMEs. Support targeted specifically at SMEs is needed because smaller centres are much more reliant on SMEs for business activity and employment, the entrepreneurial base is aging and will need to be replaced, SMEs have much more restricted access to loan financing, and rural areas tend to have more limited access to business services.

There is also a need to enhance the capacity of local and regional governments, catalyze partners, and leverage funding from other sources than the CFP to support community strategic planning and CED projects. While providing support directly to small businesses is an important strategy for promoting community economic development, many key issues are best addressed through broad systemic initiatives in partnership with local governments. CFDCs play an important role as a catalyst for such initiatives (as demonstrated in case the studies), building partnerships and bringing the capacity, expertise, support, and business perspective needed to move community plans and CED initiatives forward, particularly in regions where local organizations have limited capacity for economic development.

Relationship to Other Programming

2. The activities of the CFDCs complement rather than duplicate or overlap other resources that are available.

There are numerous other government funded programs available to businesses and smaller communities in southern Ontario which share objectives similar to those of the CFP. These programs include those which provide financing for small businesses, those which provide other services for SMEs, and those which support community strategic planning and CED initiatives. The greatest potential for overlap lies in the provision of basic business information and services as well as larger loans. The actual level of duplication and overlap is low because of the strong demand for services and the significant coordination, collaboration, leveraging of resources and differentiation that exists between the programs. The flexible design of the CFP enables CFDC activities to be tailored to the specific needs and gaps in services within the local communities.

Alignment with Federal Priorities, Roles & Responsibilities

3. The objectives of the CFP are aligned with the strategic outcomes of FedDev Ontario as well as with federal government priorities, roles and responsibilities.

The objectives of the CFP align well with FedDev Ontario's strategic outcomes, particularly with respect to sustained economic development, growth of communities and "a competitive southern Ontario economy," as well as with recent Speeches from the Throne that echo the Government of Canada's commitment to job creation, support to small businesses, and the economic well‑being of communities. The programming is also consistent with the commitment of the federal government, under the Constitution Act (1982), to promote, "economic development to reduce disparity in opportunities."

Performance – Effectiveness

Financing and Business Development Services

4. From 2009–10 to 2012–13, the CFDCs provided a range of services to approximately 2,300 loan clients including 2,779 loans valued at over $165 million.

On average, 75 loans valued at $4.5 million were issued per CFDC over the four years of the evaluation. A greater number (105 vs. 55 per CFDC) and value of loans ($5.6 million vs. $3.7 million per CFDC) were issued per CFDC by CFDCs in the East compared to those in the West. This difference can be explained, in part, by differences in population density (i.e., the Eastern Region has a lower population density which is typically associated with fewer business services), data outliers, and potential differences in levels of awareness of CFDC services.

In addition to loans, 54% of the loan clients surveyed also received one‑on‑one business counselling, 42% participated in a training program, 41% received business plan assistance, and 35% received business information. These services had a notable impact in terms of helping clients to further develop their business or management skills (reported by 34% of clients), access other programs and services (31%), improve business practices (29%), or start their own business (24%). The CFP also had a significant impact in terms of increasing access to capital; a majority of loan recipients were either turned down for funding from other sources (usually a bank or credit union) or believe it is unlikely that they could have accessed funding from other sources. Most loan clients believe that the CFDC loans and services were very important in the establishment or further development of their business.

The CFDC loans were leveraged by funding from other sources. Clients who were surveyed reported investing an average of $1.50 in funding from other sources for every $1.00 loaned by the CFDC while, in their quarterly reports, the CFDCs reported an average of $2.00 leveraged for every $1.00 loaned. The most common sources were investments by the client or partners, commercial lenders and family and friends. The loans and leveraged funds were used primarily to start up operations, fund capital or leasehold improvements, expand production, and provide working capital.

5. CFDC loan clients grew faster, had higher rates of survival, and generated an estimated additional $516 million in revenues, 3,865 jobs, and almost $130 million in wages over a five year period compared to similar businesses that did not receive loans.

The Business Number analysis using Statistics Canada data indicates that SMEs which received loans from CFDCs in southern Ontario grow significantly faster (average revenue growth of 14.9% vs. 6.9%) and have higher survival rates (88% vs. 66%) after five yearsFootnote 70 than similar businesses that did not receive CFDC loans. Using the results of the client survey and the Business Number analysis, it is estimated that, in comparison to the control group, CFDC loan clients generated an additional $516 million in revenues, 3,865 jobs, and almost $130 million in wages over a five year period.

6. CFDC clients receiving only business information and services improved their business skills and practices, started and further developed their business, and expanded their access to other programs and services. However, the impact of the services is unquantifiable due to a lack of adequate data.

Of the non‑loan clients who were surveyed, 29% received one‑on‑one business counselling, 29% received business plan assistance, 27% participated in training, and 27% received business information. These services had a notable impact in terms of helping the clients to further develop their business, management or business skills (identified by 39% of clients), encouraging them to start their own business (33%), improving their business practices (30%), and increasing access to other programs and services (25%). As demonstrated in case studies, CFDCs also helped to enhance knowledge, skills, and awareness of entrepreneurship among local youth.

Sixty‑three percent of the non‑loan clients surveyed are currently operating businesses and, on average, the revenues of businesses have increased by an average of 15% per year since the services were received. Over two‑thirds of non‑loan clients rated the services as important or very important in the further development of the business. However, there is no reliable basis on which to extrapolate the results to the total population of non‑loan clients given that there is no Business Number analysis using Statistics Canada data on the growth and survival of non‑loan clients; there are no standard parameters such as loans received on which to base the extrapolation; it is generally much more difficult for non‑loan clients than loan clients to attribute their growth to the assistance received; and there is no basis for assessing how representative the non‑client respondents are of the total population of CFDC clients who did not receive loans.

Strategic Planning, CED Projects and Partnerships

7. The CFP plays an important role in community economic development and strategic planning, particularly in terms of in‑kind support from CFDC staff (e.g., in leveraging funding from other government programs and partners, catalyzing partners, and enhancing the capacity of local and regional governments).

Almost all CFDCs (92%; 93% in the East and 91% in the West) have led CED projects over the past four years while 59% of CFDCs (60% in the East and 59% in the West) have taken the lead in the development of one or more community strategic plans. The CFDCs reported involvement in 2,317 CED initiatives, 30% of which they led. The CFDCs used staff and in‑kind resources to deliver $8.7 million for these CED initiatives (most of which was accessed through service delivery agreements with other programs and government departments), which has been further leveraged with $45.5 million in funding from other partners ($5.20 for every dollar leveraged by the CFDC). The CFDCs reported an average of three partners per CED project in which they were involved.

There have been different levels of CED activity in the Eastern and Western Regions due to differences in levels of funding available through service delivery agreement programs (i.e., funding other than the CFP), particularly the EODP and the CAF. CFDCs in both the Eastern and Western Regions have been active in community strategic planning and CED initiatives. However, CFDCs in the Eastern compared to the Western Region have been more active in CED projects as a result of receiving more CED funding from other programs (e.g., EODP funding to Eastern CFDCs totaled four times the level of CAF funding to Western CFDCs per CFDC over the four years of the evaluation).

Most key informants rate the CFDCs as successful in strengthening community capacity for socio‑economic development, by increasing access to capital, strengthening the entrepreneurial base, and further building the economic development capacity of local organizations. The case studies provide examples of how the CFDCs have strengthened community capacity by enhancing the business skills of youth, improving community infrastructure, increasing awareness and pride in the region, strengthening relationships with partners, building the capacity of the community to engage in sustainable economic development, and strengthening the capacity of social enterprises.

Ultimate Outcomes

8. Most key informants view the CFDCs as effective in supporting small business development, leveraging funding from other programs, and working with partners to promote economic stability, growth, and diversification.

According to all groups of key informants, the CFP has been particularly successful in facilitating small business development, the creation or maintenance of jobs, economic stability (by expanding the economic base), and diversification. As noted earlier, businesses have started and expanded as a result of CFP business financing and services ($516 million in new revenues and 3,865 new jobs). CFDCs worked with various partner organizations in delivering services, developing strategies and implementing CED initiatives to support economic growth, stability and diversification. As demonstrated in the case studies, CFDCs have partnered with regional governments to attract businesses to the region and have leveraged provincial and federal government funding and engaged partners to maintain and diversify jobs.

Some key informants noted that the business financing and services have tended to be less successful in supporting innovation, diversifying the economies away from traditional industries, and improving competitiveness compared to efforts aimed at structural, long‑term change such as CED and community strategic planning.

Contributing and Constraining Factors

9. The major factors which contribute to the effectiveness of the CFP include the flexible, locally based delivery model, the experienced and knowledgeable CFDC staff and board members, and the strong connection of the CFDC to the community.

A number of factors contribute to the effectiveness of the CFP. Key informants attributed the effectiveness of the CFDCs to the flexible, locally‑based delivery model; experienced, knowledgeable and dedicated staff and board, strong connections within the community; the high level of accountability; familiarity with local needs, opportunities and stakeholders; and the extent to which the services of the CFDCs are integrated with those of other organizations. Overall, CFDC clients are satisfied with the services they have received. Clients noted that the CFDC staff are professional, supportive, helpful, knowledgeable, respectful and responsive to their needs, the services met their needs, and the services (particularly the loans) were not available from other sources.

10. The main factors which constrain the effectiveness of the CFP include internal factors such as the level of funding for CFDC operations, and external factors such as the 2008 economic recession, the challenges associated with the small size of communities, and a lack of willingness of some governments or organizations to partner with the CFDC.

Both internal and external factors constrain program effectiveness. The level of funding available to CFDCs constrains their ability to deliver services and dedicate staff time to support community strategic planning, CED projects and partnerships. No funding is specifically set aside for CED projects or community strategic planning and, until the current fiscal year, the level of funding provided to individual CFDCs was not tied directly to program activities or reported outcomes. Other internal constraints include confusion regarding the objectives of the program, the amount of CFDC staff time dedicated to CFP reporting, and a lack of clarity around the roles of the four Community Futures associations.

External constraints include economic conditions (e.g., the 2008 economic recession), challenges associated with the small size of communities (e.g., limited transportation, telecommunications and municipal infrastructure), and a lack of willingness among some local governments or organizations to partner with the CFDC (e.g., lack of awareness of the CFDC, more focused on own community's growth instead of regional goals, etc.). CFDCs have implemented various strategies to mitigate internal and external barriers, such as changes to their lending activities and services, increased partnering, and changes to their delivery model, among others.

Performance Measurement

11. While extensive performance information is collected and reported and best practices have been developed (e.g., Business Number analysis using Statistics Canada data), there are issues associated with performance measurement and reporting that need to be addressed, such as the accuracy and utility of some measures, the consistency in reporting across CFDCs, and the level of effort required.

FedDev Ontario collects extensive information on a quarterly and annual basis from the CFDCs to meet the requirements of the Performance Measurement Strategy and for planning and reporting purposes. In addition, the Business Number analysis using Statistics Canada data, which compares the performance of CFDC clients and a similar group of businesses, was identified as a best practice in performance measurement in terms of data accuracy, attribution and analysis of impacts. However, there are some significant issues associated with performance measurement that need to be addressed, including:

Performance – Efficiency and Economy

Efficiency

12. The CFP service delivery model is an efficient and effective approach to community economic development. The local and flexible delivery structure allows CFDCs to focus on local needs and build trust with partners and businesses.

The CFP service delivery model, which is in place across Canada, is efficient and effective because it allows CFDCs to respond to local needs at a grassroots level. Strengths of the model include local delivery, independence from government, the flexible delivery structure, the integrated services (i.e., business financing and counselling, support for CED projects, and support for community strategic planning), extensive use of partnerships, the long history of the CFDCs in the communities (which has enabled them to build trust and expertise), and the strong commitment of the CFDC volunteers and staff.

13. Even if only the impacts generated from services provided to loan clients are considered, the CFP generates a strong return on investment for every dollar invested annually in program grants and contributions.

The CFP generates a strong return on investment for the Government of Canada in terms of the grants and contributions costs per job created. Based on Business Number analysis using Statistics Canada data, for every dollar contributed (based on an average annual CFP grants and contributions budget of $11.8 million), the CFP in southern Ontario generated $43.76 in business revenues and $10.97 in wages, and cost $3,053 per job created in the five year period since loans were received. The costs per job are comparable to other programs with similar objectives (e.g., FedDev Ontario's CME‑SMART Program resulted in an average cost of $7,751 per job created and $3,545 per job maintained). Furthermore, the figures significantly understate the impacts of CFDCs since they do not consider the impacts of services provided to CFDC clients that did not receive a loan (e.g., received only counselling services or training) or for the contribution of CFDCs to CED initiatives and community strategic planning activities.

14. There is room to improve the efficiency of the delivery of the CFP in southern Ontario since costs tend to be higher for some outputs in southern Ontario than in Western and Atlantic Canada, efficiency varies widely across CFDCs, and the roles and efficiency measures of the Community Futures associations could be better defined.

For some outputs, the delivery of the CFP in southern Ontario is less efficient than Western and Atlantic Canada, particularly in terms of budgeted costs per CFDC staff member and loan issued. However, southern Ontario tends to have lower costs per job created or maintained and leverage more business financing per dollar cost. The differences are due, in part, to differences in the types of services delivered and how data is collected and reported. There is also a significant variation in costs per outcome across southern Ontario CFDCs, which could also indicate varied levels of efficiency or varied demand for services. In addition, while the CFDCs are generally satisfied with the services provided by the four Community Futures associations, no clear performance indicators are in place to assess the effectiveness, efficiency or economy of their services.

Economy

15. While the past four years have been a transition period for the administration of the CFP in southern Ontario and a number of activities have been undertaken to optimize delivery, the level of funding, new funding model, and governance challenges may constrain the achievement of program outcomes.

FedDev Ontario has been working to optimize program outcomes since it took over the administration of the CFP in southern Ontario in 2009. Activities have included an organizational review of all 37 CFDCs in 2010, developing a funding model which incorporated Key Performance Indicators (KPIs) in 2011, holding engagement sessions with CFDCs and stakeholders in 2012 to develop and define performance indicators, and implementing a Performance Based Funding Model in 2012 to seek further optimization within a reduced program budget. Though it is too early to determine the impacts of these changes, there is some concern that the reduced level of guaranteed funding and uncertain level of performance based funding will constrain CFDC delivery of services and may discourage CFDCs from addressing CED and community strategic planning needs in their communities (e.g., only 10% of outcome‑based funding pertains to CED and strategic planning performance). Incentives under the Performance Based Funding Model could also affect loan decisions, influence how activities and outcomes are tracked and reported, impair the ability of CFDCs to attract or retain staff, or add to the complexity of making longer‑term strategic decisions. Challenges with respect to FedDev Ontario governance of the program may also constrain outcomes (e.g., a lack of clarity in program objectives, changing and duplicative reporting, delays in contract approvals and auditing for claims, inflexible deadlines, and too much oversight of CFDCs).

Management of Loan Funds

16. Though there is some variation across individual CFDCs, on average, the CFP loan funds are well managed. Over the four years, 71% of the funds were in active loans and funds grew by 7%. In addition, the lending limits and loan loss rates are appropriate.

On average, the CFDC investment funds are well utilized. Over the four year period, an average of 71% of the investment funds were in active loans (the average utilization was 71% in both the Western and Eastern Region). The total value of the investment funds grew by 7% (approximately $12 million) from 2009–10 to 2012–13, as a result of positive returns earned on the loans. Recapitalizations through CAF contributed to an additional increase of $29 million. In addition, key informants perceived that the loan loss rate (average of 4.5%) and lending limit ($150,000 and $250,000, on an exceptional basis) are appropriate. However, there may be opportunities to further optimize the use of loan funds in some regions since some funds were almost completely utilized in active loans, while other funds were only about half utilized (e.g., increase flexibility through regional loan pools to allow CFDCs with high demand for investment funds to access under‑utilized investment funds in regions with lower demand).

B. Recommendations

The major recommendations arising from the evaluation of the Community Futures Program in southern Ontario are as follows:

1. Pursue opportunities to develop further partnerships and increase collaboration with other organizations in order to leverage CFP services (i.e., business services and lending and support for CED projects and community strategic planning).

With a local presence and a flexible structure which allows services and activities to be tailored to local needs and conditions, the CFDCs are well‑positioned to facilitate a coordinated and integrated approach to both small business development and community economic development. While recognizing the record of CFDCs in developing strong relationships with other organizations, there are opportunities for CFDCs to further:

FedDev Ontario should work closely with other federal and provincial government programs and services to further coordinate and integrate services. While coordination can and does occur at the local level, there are also opportunities to further coordinate and even consolidate programs and services at the government level. Coordination between the CFP and other programs, particularly those funded and delivered through the provincial government, could improve CFP effectiveness, efficiency, and economy.

2. Build upon performance measurement efforts by addressing attribution and consistency challenges, reducing duplication, monitoring performance measurement systems, and working with other RDAs to share best practices.

FedDev Ontario has an opportunity to build upon performance measurement activities that have been undertaken over the past four years (e.g., CFDC organizational review, consultations to develop and define indicators, development of the Performance Based Funding Model, etc.) and to build upon the mechanisms that have been developed to track and measure CFP performance in order to further strengthen CFP performance data. Specific areas include:

3. Continue to include CED and community strategic planning as core program activities and facilitate CFDC involvement in these areas by supporting CFDCs in leveraging funding from other sources, ensuring the activities are sufficiently resourced, and reducing the reporting burden on CFDC staff time.

CFDCs play a key role in providing in‑kind and staff support to catalyze and leverage funding for CED and community strategic planning. As described in the first two recommendations, FedDev Ontario can support CFDC involvement in these areas by encouraging CFDCs to leverage funding from other sources and catalyze partnerships, ensuring the activities are sufficiently resourced (either through Performance Based Funding or other funding mechanisms) while taking into account contextual differences across regions (e.g., access to EODP funding in the Eastern Region), and by reducing the time burden on CFDCs associated with reporting (e.g., reduce the number and frequency of reports, develop mechanisms for increased automation, etc.) so that CFDC staff have more time to address strategic planning and economic development needs in their communities.

With respect to the differences in access to CED and planning project funding across regions and the potential need for additional funding, a further assessment of the relative need and impact of such funding should be determined as part of evaluations of other programs administered by FedDev Ontario (e.g., EODP and CAF).

4. More clearly define the roles (e.g., marketing of the CFP, information sharing and support), mandates, and performance measurement requirements for the four Community Futures associations to improve responsiveness and accountability to CFDC needs.

Community Futures associations play an important role in supporting CFDCs, which can be further defined to ensure they are responsive to CFDC needs and accountable to the funding they receive. Associations play a key role in areas such as professional development, identification and communication of best practices, promoting coordination and collaboration across CFDCs and with other organizations, and other support services. Associations could also play a role in further enhancing marketing and branding of the CFP across southern Ontario, raising the public and political profile of the CFP while preserving the ability of local CFDCs to tailor services to local needs. Though there is no consensus regarding how the associations should be structured and funded, FedDev Ontario should consider different options with respect to funding the Community Futures associations such as reducing the number of associations funded, encouraging amalgamation or greater coordination of activities, or decentralizing the funding decisions to the CFDCs. In addition, clearer performance indicators and measures should be developed to encourage greater responsiveness and accountability. Indicators could include CFDC satisfaction levels (e.g., survey of CFDC members), program performance (e.g., of SOFII), participation levels (e.g., proportion of CFDCs that are active members or served by the organizations), and activity levels (e.g., number of training workshops organized, participants at workshops, networking events hosted, or best practices information notices issued).

5. Strengthen communication and partner relationships with CFDCs.

Recognizing that the past four years represent a transition period for the administration of the CFP under FedDev Ontario, the Agency should continue to work to strengthen communication and its partner relationships with individual CFDCs. FedDev Ontario should provide clear and consistent direction with respect to program objectives around CED and community strategic planning, and expectations regarding lending, business counselling, acceptable loan loss rates (e.g., based on national CFP loan loss rates), and appropriate levels of risk. Project officers play an important role in strengthening the partnership between FedDev Ontario and CFDCs. Project officers should have more face‑to‑face meetings with CFDCs, increase their awareness of CFDC regional needs, and work with CFDC management and Boards to engage local governments and to facilitate partnerships and collaborations. FedDev Ontario could further capitalize on CFDCs and their knowledge of local needs and presence in the community to leverage other FedDev Ontario programming and further Agency objectives.

top of page

Appendices

Appendix I: overview of feddev ontario

Table 17: overview of feddev ontario
Program Activity Budget
($ million)
Sub-activities Key Indicators of Performance
Business Development $118.5
  • Investing in Business Innovation
  • Prosperity Initiative
  • Business Productivity and Innovation
  • Increase in business investment in machinery and equipment in Ontario
  • Increase in real hourly wages in southern Ontario
  • Increase in labour productivity in Ontario businesses
Technological Innovation $51.0
  • Graduate Enterprise Internship,
  • Scientists and Engineers in Business
  • Youth STEM
  • Applied Research and Commercialization
  • Technology Development Program
  • Increase in the number of employees in southern Ontario that are considered "highly skilled and qualified personnel"
  • Increase in the amount of investment in research and development by Ontario businesses
Community Economic Development $34.5
  • Community Futures Program
  • Eastern Ontario Development Program
  • Official Language Minority Communities
  • Infrastructure Delivery (e.g., Building Canada Fund)
  • Number of business in southern Ontario communities that are in "growing" industries
  • Increase in the southern Ontario employment rate
Internal Services $14.8
  • Management and Oversight
  • Communications
  • Legal
  • Human Resources Management
  • Financial Management
  • Information Management
  • Information Technology
  • Real Property
  • Materiel
  • Acquisition
  • Travel and Other Administrative
Total $218.8
  • Increase in Ontario Gross Domestic Product in comparison to similar economies
  • Increase in the Ontario employment rate in comparison to similar economies

Source: Federal Economic Development Agency for Southern Ontario 2012–13 Report on Plans and Priorities.

Appendix II: evaluation issues and questions

Table 18: evaluation issues and questions
Evaluation Issues: Evaluation Questions:
Relevance
  • 1.1 Is there a continued need for business financing, business information and services, community strategic planning, and community economic development (CED) projects and partnerships supported by the CFP?
  • 1.2 To what extent are the objectives of the CFP aligned with departmental strategic outcomes and federal priorities and strategies?
  • 1.3 To what extent are the objectives of the CFP aligned with the federal government's activities, roles and responsibilities?
  • 1.4. Does the CFP complement, duplicate or overlap other government programs or private sector services?
Performance - Effectiveness
  • 2.1. What impact would the absence of CFP funding have had on the start-up, survival and growth of businesses, and on community strategic planning and development?
  • 2.2. To what extent have the CFP activities contributed to improved access to capital and enhanced business services?
  • 2.3 To what extent have the CFP activities contributed to strengthened community strategic planning and more effective implementation of CED through projects, partnerships and CED initiatives?
  • 2.4 To what extent have the CFP activities and immediate outcomes contributed to improved business practices, increased entrepreneurship, and strengthened and expanded businesses?
  • 2.5 To what extent have the CFP activities and immediate outcomes contributed to strengthened capacity for socio-economic development?
  • 2.6 To what extent have the CFP activities, immediate and intermediate outcomes contributed to its identified ultimate outcomes of economic stability, growth and job creation, diversified and competitive local rural economies, and economically sustainable communities?
  • 2.7 What are the barriers to achieving the CFP immediate, intermediate and ultimate outcomes and to what extent are these being mitigated?
  • 2.8 To what extent are the CFP's performance measurements and reporting structures effective in reporting on the achievement of the CFP's outcomes? How is the performance information used by the CFDCs and FedDev Ontario?
Performance – Efficiency and Economy
  • 3.1 To what extent is the CFP efficient in the context of the results being achieved?
  • 3.2 Is there a more cost effective way of achieving expected results, taking into consideration alternative delivery mechanisms, best practices and lessons learned?
  • 3.3 Are the CFDC investment funds well managed? Are the loss rates of the loans acceptable? Do the CFDCs carry an acceptable level of risk? Is the current lending limit of $150,000 in the CFP Terms and Conditions appropriate?

Appendix III: evaluation methodology

A. evaluation approach and methodology

The evaluation used multiple lines of evidence including primary data sources (surveys of clients, CFDC representatives and community partners as well as interviews with representatives of FedDev Ontario and other stakeholders), secondary data sources (review of administrative data, documents, literature, and Statistics Canada data analyses), and case studies which draw from both primary and secondary sources. Each data source is described in more detail in the following paragraphs.

1. Survey of Clients

The analysis includes the results of a survey of 512 CFDC clients, including 309 loan clients and 203 non-clients. To survey the clients in southern Ontario, we:

As indicated, a maximum of 1,290 loan clients and 2,130 non-loan clients were contacted (these figures are significantly overstated in that we cannot assume, just because an email did not bounce back, that it was necessarily received and viewed by the target recipient). As indicated in the table below, 26% of the loan clients and 11% of the non-loan clients who may have been reached responded to the invitation to participate in the survey. The response rate of 16% for the survey overall and the 11% response rate for the non-clients is significantly understated to the extent that many of the non-loan clients targeted were likely not eligible to participate in the survey because they had received little in the way of services or could not recall, in much detail, the services that they received in the past.

Table 19: response rate by type of client
Province Loan
Clients
Non-loan
Clients
Total
Apparent Number of Clients Contacted
Unique Clients 1,971 2,798 4,769
Clients With contact information including email 1,669 2,581 4,250
Not contacted (i.e., email bounced and no telephone number, wrong number or not in service, or otherwise couldn't contact) 379 451 830
Maximum Number of Clients Contacted 1,290 2,130 3,420
Response Number % Number % Number %
Completed 291 23% 192 9% 483 14%
Substantially completed 18 1% 11 1% 29 1%
Respondents Included in Analysis 309 24% 203 10% 512 15%
Begun but not substantially completed (results not included in analysis) 28 2% 12 1% 40 1%
Total Number of Respondents 337 26% 215 11% 552 16%
Refused or unsubscribed 161 12% 286 13% 447 13%
Did not respond 792 61% 1,629 76% 2,421 71%
Total 1,290 100% 2,130 100% 3,420 100%

At a confidence level of 95%, the sample of 512 Community Futures clients who completed or substantially completed the survey (at least four of the six sections) achieves a margin of error of about ±4%. At the level of 95%, the confidence interval for the survey of loan clients is ±4.8% while the confidence interval specifically for the non-loan clients is ±6.5%. Of the 512 surveys completed, 146 were completed by telephone and 366 were completed online.

The survey data was then linked with the sampling records obtained from the CFDCs, which enabled certain gaps in the responses to be filled (e.g., value of loans received) and to compare the growth of the business to the projected number of jobs that would be created.

2. Survey of CFDC Representatives

A survey of 79 CFDC representatives (staff and board members of CFP-funded organizations; e.g., CFDCs) was conducted to collect information on the activities and outcomes of CFP-related programming. Representatives were surveyed from all 37 CFDCs including 36 general managers, executive directors and CEOs, 33 board members, 6 loans managers/officers, 1 business services officer, 2 community economic development coordinators, and 1 administrator. On average, the managers have been involved in the CFP for 12 years.

3. Survey of Community Partners

A survey was conducted with 73 representatives of community organizations that are involved in or affected by the delivery of the CFP activities and are in a position to comment on the impact of the program in their respective communities. On average, the community partners have been involved with the CFDCs for 9 years, in various capacities including partnering with the CFDC on CED initiatives and the development of community strategic plans, referring clients to the CFDC or receiving referrals from the organization, and serving as CFDC Board members or members of the loans committees.

Table 20: involvement of community partners with the CFDCs
Involvement with the CFDC Number %
Total Number of Community Partners Surveyed 73 100%
Partnered with the CFDC in implementing one or more community economic development (CED) initiatives 70 96%
Refer clients to the CFDC 51 70%
Partnered with the CFDC in the development of one or more community strategic plans 40 55%
Receive referrals of clients from the CFDC 30 41%
Has been a member of the Board 15 21%
Has served on the loans committee 7 10%
Other (collaborated in the areas of marketing, client services, or community planning) 5 7%
4. Interviews with Representatives of FedDev Ontario

Extensive telephone interviews were conducted with 12 representatives of FedDev Ontario including four senior managers and executives, three managers, four program officers, and one claims officer. Of the 12 representatives, eight were located at FedDev Ontario Headquarters in Kitchener (Western Region), three were located in the Peterborough office (Eastern Region), and one was located in Ottawa. The interviews provided a unique perspective of the program from individuals who are closely involved in design and delivery of the program. Their roles with respect to the CFP are summarized in the following table.

Table 21: involvement of feddev ontario representatives in the CFP
Position Roles and Responsibilities Related to CFP
Senior Executives
  • Director General of Innovation and Economic Development
  • VP of Business, Innovation and Community Development
  • VP of Policy, Partnerships & Performance Management (policy oversight)
Managers
  • Oversee program officers in East and West delivery regions
  • Oversee policy group in Kitchener Headuarters office
Program Officers
  • Work directly with CFDCs (Western office oversees 22 CFDCs; Eastern office oversees 15 CFDCs)
  • Develop a contract and agreement with CFDC (provide advice and guidance)
  • Oversee CFDC operations (CFDCs report on loans, activities, business plan, investment fund)
  • Administer grants and contributions for CFP
  • Some responsible for CFDC associations (e.g., OACFDC, WOCFDCA)
Claims Officers
  • Screen and review all the costs
  • Review costs submitted
  • Screen for eligibility against agreements

The interview guide was structured to obtain input regarding program relevance, alignment with departmental and government priorities, achievement of expected outcomes, and cost-effectiveness of the program activities. On average, the FedDev Ontario representatives have been involved in the CFP for five years.

5. Interviews with Other Stakeholders

Interviews were conducted with 16 other stakeholders who are familiar and have some interaction with the CFDCs but are not directly involved in the design and delivery of the program. These stakeholders included representatives of two federal government agencies (Agriculture and Agri-Food Canada and BDC), seven representatives of provincial government agencies (Ministry of Economic Development and Trade and Ministry of Rural Affairs), and seven representatives from other organizations (CYBF, Ontario Chamber of Commerce, OACFDC, EOCFDC Network, and WOCFDCA).

The representatives are familiar with the CFP (providing an average familiarity rating of 4.2 on a scale of 1 to 5, where 1 is not at all familiar and 5 is very familiar). As indicated below, the most common interaction involves referrals clients between their organization and the CFDCs.

Table 22: involvement of other stakeholders with the CFP
Involvement Number Percent
Number of Other Stakeholders Interviewed 16 100%
Refer clients to the CFDC 11 69%
Receive referrals of clients from the CFDC 10 63%
Activity specific interaction (e.g. partnering on co-lending and service delivery, networking, and liaison on specific projects) 10 63%
Have partnered in the development of community strategic plans or community economic development (CED) initiatives 9 56%
Have been a Board member of a CFDC 1 6%
6. Document and Literature Review

An extensive review of documents, files and Statistics Canada data was conducted with a particular focus on obtaining information pertaining to relevance, cost effectiveness and impact of the program. The review included all relevant internal (both Agency-wide and region specific) and external documents, including documentation related to the CFP (e.g., Business Number analysis conducted by Statistics Canada comparing the performance of CFP-assisted firms to a similar group), FedDev Ontario, and federal government policies and strategies and other documents deemed relevant to the programming (e.g., the Performance Measurement Strategy and past evaluations). External documents related to CED programming and the context in which the CFP operates were also reviewed (e.g., community strategic plans and documents on CED initiatives). The literature review focused primarily on federal policies and strategies, other similar programs operating in southern Ontario and jurisdictions, and economic data (e.g., population, labour force participation rates, employment, unemployment rates, and household income) related to the region served by CFDCs. A list of databases, documents, and literature reviewed is provided in Appendix I.

7. Administrative Data

A detailed analysis of project data from 2009–10 to 2011–12 and, where feasible, 2012–13 was undertaken to create a profile of activities implemented through the program. The administrative data provides a comprehensive perspective of the outputs and activities of the program (such as statistics on program activities, loans, business support services provided, etc.). The data is collected by FedDev Ontario on quarterly and/or annual basis. Project data was supplemented by resource data from FedDev Ontario's financial and human resource systems where possible.

8. Case Studies

Four CFDC regions were selected for in-depth case studies based on a variety of criteria including location, population, population density, Aboriginal population, level of unemployment, involvement in CED projects, and involvement in unique, innovative or otherwise promising practices identified during key informant interviews. Each case study involved:

The case studies also provided evidence on promising practices and lessons learned with respect to the delivery of the programming and projects. The CFDCs and the initiatives or models selected for case study are presented in the table below.

Table 23: communities selected for case studies
CFDC Topic of the Case Study
Trenval Business Development Corporation The purpose of this case study was to examine the partnership model Trenval has developed in its co-location with several economic and business development organizations, and how this model enhances coordination and support for business and economic development in the region.
Frontenac Community Futures Development Corporation The purpose of this case study was to learn about Frontenac CFDC's food and beverage promotion activities and how the activities have benefited business and economic development in the region.
Nottawasaga Futures The purpose of the case study was to examine the Green Economy Centre and how the Centre has supported businesses to reduce costs, maintain jobs and grow.
Sarnia–Lambton Business Development Corporation The purpose of the case study was to assess the Aboriginal Youth Entrepreneurship Program and how it has helped to increase awareness of entrepreneurship, and build business skills among Aboriginal youth participants.

Site visits were made to two CFDC communities: the Trenval Business Development Corporation office in Belleville, Ontario and the Nottawasaga Futures office in Alliston, Ontario. Each site visit included:

The case studies in the two communities where site visits were not made involved a document review, interviews with representatives involved in the project or model selected for review (i.e., CFDCs, community partners, and clients involved in the project or model selected for case study), and an analysis of the surveys completed in that region. As indicated below, 33 representatives provided input into the four case studies and a total of 110 surveys and interviews were included in the analysis.Footnote 71

Table 24: number of focus group participants, surveys and interviews conducted in each case study region
Key Informants Trenval Nottawasaga Frontenac Sarnia–Lambton Total
Focus Group Participants and Case Study Interviews
CFDC Representatives 3 4 1 4 12
Community Partners 7 1 5 1 14
CFDC Clients - 3 4 - 7
Total 10 8 10 5 33
Surveys and Interviews
Clients 25 32 31 7 95
CFDC Representatives 4 3 1 2 10
Community Partners 2 1 2 - 5
Total 31 36 34 9 110

The data was compiled, analyzed, and synthesized according to the major findings for key evaluation indicators. A summary of the case study findings is provided in Appendix II (in a separate document).

B. challenges and limitations

The challenges and limitations associated with this evaluation included the potential for respondent bias and non-response error. Many key informants (CFDC representatives, FedDev Ontario representatives and community partners) and clients are involved in program design and delivery or are direct beneficiaries who have a vested interest. Consequently, the findings may be biased towards more favourable program outcomes. Several measures were taken to reduce the effect of respondent biases and validate the survey results: (i) the purpose of the surveys and interviews, its design and methodology, and strict confidentiality of responses was communicated clearly to participants; (ii) answers were cross-checked from each sample of respondents with the other groups for consistency and validation; and (iii) wherever possible, the findings from surveys and key informant interviews were triangulated and validated with the results of the document and administrative review as well as the case studies.

Factors that contributed to the potential for non-response error among clients included issues associated with population list (e.g., completeness and currency of the contact information provided by the CFDCs), the length of time since the services were provided to the clients (up to four years ago which may impact on the availability of clients as well as their willingness and ability, particularly of non-loan clients, to respond), and differences in the level of service provided and outcomes (clients who received more significant services such as large loans and who are currently operating are more likely to respond to the survey). To mitigate the potential effects, three email reminders and several follow-up phone calls were made to clients with a particular emphasis on regions which were under-represented in the initial response. In the end, the respondents were relatively evenly divided between those in those in the Western Region (259) and those in the East (253).

To assess the non-response error, the characteristics of those surveyed were compared with clients in the sampling frame to assess consistency based on the data currently available. The characteristics of respondents varied from those of the total population in terms of gender, incidence of loan clients, and average size of loan.

The completion rate for the survey varies somewhat across questions. The financial questions asked of clients (e.g., questions about the revenues generated by the business in the year before receiving assistance and in the latest year) tended to have a lower response because some clients, particularly non-loan clients, are hesitant to share financial data or could not recall financial data for past years. Of the 309 loan clients, 239 provided financial data, 20 are not operating, 27 are still in their first year of operations, and 23 declined to provide financial data. Of the 203 non-loan clients, 74 provided financial data, 72 are not operating, 32 are still in their first year of operations, and 25 declined to provide data. To reflect this challenge, the number of clients responding to any particular question is included in the tables throughout this report.

top of page

Appendix IV: References

Databases:

Documents:

Literature:

Appendix V: community rutures program logic model

Figure 2: Community Futures Program Logic Model

Graph of Community Futures Program Logic Model (the long description is located below the image)
Description of Figure 2

Figure 2 is the Logic Model for the Community Futures Program.  A logic model shows how program inputs are used to conduct activities which are expected to produce outputs and in turn how these outputs are expected to lead to different levels of outcomes.

The inputs for the Community Futures Program are program financial resources for operations and maintenance, salaries and wages and grants and contributions. These inputs are used to fund two categories of activities: 1) development/maintenance; and, 2) program delivery.

The development/maintenance activities include:

  • program development, planning and program management;
  • monitoring and providing non-financial support to Community Futures organizations; and
  • measuring community Futures organization performance and allocating funding.

These development/maintenance activities are expected to lead to the following Industry Canada outputs respectively:

  • policies, plans, reports and program tools, and resources;
  • advice, information and support; and,
  • funding adjustment.

The sole program delivery activity is providing funding to Community Futures organizations. This activity, in conjunction with the development/maintenance activity, advice, information and support, are expected to lead to the Industry Canada output of capitalization and operating contributions with Community Futures organizations to support community economic development through strategic community planning and implementation, business services and access to capital. This Industry Canada output is expected to lead to the following Community Futures organization outputs:

  • business financing;
  • business development knowledge/skills;
  • community strategic plans; and,
  • community economic development projects and partnerships.

It should be noted that the last Community Futures organization output is also partially expected to be due to the development/maintenance Industry Canada output, funding adjustment.

The Community Futures organization outputs listed above are expected to lead to the following immediate outcomes respectively:

  • improved access to capital and leveraged capital through loans, loan guarantees and equity investments to businesses and social enterprises;
  • enhanced/maintained business development services – information, counselling, referrals and training;
  • strengthened community strategic planning; and,
  • more effective implementation of community economic development through projects, partnerships, and other community economic development initiatives.

The four immediate outcomes, in combination, are expected to three following intermediate outcomes:

  • improved business practices and increased entrepreneurship;
  • strengthened and expanded businesses; and,
  • strengthened community capacity for socio-economic development.

The three intermediate outcomes, in combination, are expected to lead to the following three ultimate outcomes:

  • economic stability, growth and job creation;
  • strengthened and expanded businesses; and,
  • economically sustainable communities.

Appendix VI: performance based funding model

Overview of the Performance Based Funding Model (PBFM):

Under the Performance Based Funding Model (PBFM) individual CFDC funding is determined based on:

  1. Core CFDC Funding
    1. Base Funding: $150K to each CFDC
    2. Geographic Area Funding: Pro‑rated geographic funding contribution
    3. Official Languages Funding: $20K for each of the six designated bilingual CFDC
  2. Outcome Based Funding
    • 38% of the Agency's total CFDC funding budget will be allocated based on outcomes over the previous 3 years

      Outcome Based Funding Allocation:

      1. Loan Performance
      2. Community Partnerships
      3. Businesses Assisted
      4. Jobs
  3. KPIs Performance Contributions
    • CFDC performance will continue to be assessed against individual CFDC annual KPI targets (maximum performance contribution is $40K), paid out in the first quarter of the fiscal year and based on their performance in the previous year.

With respect to the base funding for each of the CFDCs, the Agency implemented 2013–14 as a transition year before full implementation to ease the transition to the full Performance Based Funding Model; in 2013–14, base funding for each CFDC was $225,000; in 2014–15, the base funding will be reduced to $150,000.

A. Core CFDC Funding under the Performance Based Funding Model

The core CFDC funding under the Performance Based Funding Model will be determined as follows:

  1. Base Funding: $150,000 to each CFDC
    • Transition to occur over two years; 2013–14 base will be $225,000
  2. Geographic Area Funding: Pro‑rated geographic funding contribution
    • Satellite office funding provided to two CFDCs ($75,000 total) is eliminated
    • $200,000 is available to assist service delivery in above‑average sized CFDC service areas
    • Funding will range from $8,263 to $22,752, with 16 CFDCs receiving funding
    • The two CFDCs with satellite funding were provided transition relief in 2013–14 of half their satellite funding plus their eligible geographic funding
  3. Official Languages Funding: $20,000 for each of the six designated bilingual CFDCs

B. Outcome‑Based Funding Indicators under the Performance Based Funding Model

The Outcome‑Based Funding allocation to each CFDC will be determined as follows:

Weighting Over Prior Three Years
Indicator/Metric Performance Weighting
A. Loan Performance
Total $ value of active loans 18%
# of loans in portfolio 5%
# of closed deals 16%
$ value of closed deals 5%
B. Community Partnerships
Leverage ratio 10%
C. Businesses Assisted
# of businesses assisted (services) 18%
# of businesses assisted (loans) 5%
D. Jobs
# of jobs created (loans and services) 18%
# of jobs maintained (loans and services) 5%
Total 100%
How the Model Works:

C. Key Performance Indicators (KPIs) Under the Performance Based Funding Model

The Key Performance Indicator funding allocation to each CFDC will be determined as follows:

Strategic Community Planning & Priorities
Community Economic Development Projects
Business Services
Business Loans

KPI funding is based on shares allocated based on achieved performance. Achievement of KPI targets entitled a CFDC to the following: 60% = 1 share; 80% = 2 shares; and, 100% = 3 shares. A KPI Funding Pool has been established within the CFP budget for $575,000. This funding pool is divided by the number of eligible shares to determine the value of an individual share.

KPI funding in 2013–14 based on 2012–13 performance continued under the new model with the following results:

top of page

Management Response and Action Plan

Management Response and Action Plan
Recommendation Management Response and Planned Action Management Accountability Action Completion Date

Recommendation 1

Pursue opportunities to develop further partnerships and increase collaboration with other organizations in order to leverage CFP services (i.e., business services and lending and support for Community Economic Development (CED) projects and community strategic planning).

Agreed. FedDev Ontario will continue to develop partnerships and collaborations with federal departments, provincial government ministries, and other economic development organizations. These efforts will include:

  • Working with federal organizations, such as Citizenship and Immigration Canada and the Business Development Bank of Canada, to identify potential collaborations that support the development of the rural southern Ontario economy;
  • An assessment of the benefits and best practices of the Venture Niagara Community Futures Development Corporation-St. Catharines Small Business Enterprise Centre pilot project for possible expansion to other areas; and
  • Working with the Western Ontario CFDC Association (WOCFDCA), and the Eastern Ontario CFDC Network Inc. (EOCFDC Network) to develop financing vehicles and mechanisms to ensure under-utilized investment fund capital is available and accessible within the southern Ontario Community Futures Development Corporations (CFDC) network.

Director, Community Economic Development

Identify potential collaborations and partnerships by March 31, 2015.

Complete assessment of pilot project by March 31, 2015.

Identify potential financing vehicles by March 31, 2015.

Recommendation 2

Build upon performance measurement efforts by addressing attribution and consistency challenges, reducing duplication, monitoring performance measurement systems, and working with other Regional Development Agencies (RDAs) to share best practices.

Agreed. FedDev Ontario will work with WOCFDCA and EOCFDC Network representatives to:

  • Identify opportunities to streamline and reduce duplication of CFP reporting, and integrate future reporting within FedDev Ontario's risk‑based monitoring framework; Ensure that data collected for key performance indicators is reliable and consistent across the CFDCs; and
  • Assess the preliminary impacts of the Performance‑Based Funding Model (PBFM) to identify further refinements.

In addition, FedDev Ontario will provide an update to the other RDAs on southern Ontario's key performance indicators and performance based funding best practices to inform national CFP policy discussions.

Director, Community Economic Development

Senior Manager, Program Services

Update CFP reporting requirements and definitions by December 31, 2014.

Communicate CFP performance results to Community Futures associations quarterly.

Assessment of PBFM impacts and update to RDAs by June 30, 2015.

Recommendation 3

Continue to include CED and community strategic planning as core program activities and facilitate CFDC involvement in these areas by supporting CFDCs in leveraging funding from other sources, better linking these activities to CFDC performance funding, and reducing the reporting burden on CFDC staff time.

Agreed. To further advance CED initiatives, FedDev Ontario will actively encourage the WOCFDCA and EOCFDC Network to support their members in leveraging other sources of funding, such as FedDev Ontario's Investing in Regional Diversification initiative, Eastern Ontario Development Program and Economic Development Initiative, and the Government of Ontario's Southwestern Ontario Development Fund, the Eastern Ontario Development Fund, and the Rural Economic Development program.

Director, Community Economic Development

Provide technical program briefings at the Ontario Association of CFDCs (OACFDC) Annual General Meeting in September 2014.

Recommendation 4

More clearly define the roles, mandates, and performance measurement requirements for the four Community Futures associations to improve responsiveness and accountability to CFDC needs.

Agreed. FedDev Ontario will work with the OACFDC, the WOCFDCA, and the EOCFDC Network to align roles and responsibilities and develop relevant performance metrics.

Director, Community Economic Development

Define roles, responsibilities and performance metrics by March 31, 2015.

Recommendation 5

Strengthen communication and partner relationships with CFDCs.

Agreed. FedDev Ontario will continue to strengthen communication and related activities; and partnerships with the CFDCs and the associations by ensuring that all CFDCs receive FedDev Ontario programming announcements and have access to Canada Business Ontario information and services such as the Business, Government Services and You (BGSY) events.

Director, Community Economic Development

Director, Canada Business Ontario

Deliver 6‑7 BGSY events by March 31, 2015.

top of page

Date modified: