Monthly Archives: November 2008

Completed Research Study

Standard
Case Study on Access, Quality, Effectiveness of Technical Support to Global Fund Grant Implementation at Country Level
(first 3 sections of the Study Manual)


Brief Note on the Study

Note: Consultants may use the contents of this page as introductory material to study respondents, stakeholders and others.
Introduction
UNAIDS has initiated a Case Study on Access, Quality, Effectiveness of Technical Support to Global Fund Grant Implementation at Country Level. This study is expected provide clear and practical advice for policy and programming changes that will improve technical support provision in different settings. The study will be conducted simultaneously in a sample of 9 countries and will collect information related to technical support to planning, managing implementing, monitoring and evaluating AIDS prevention, care, and support and treatment efforts in the country.
Coverage
Initiated by UNAIDS, the study would examine Technical Assistance and Support to Global Fund grants provided by all players to the National AIDS Programmes in the sample countries. The outputs of the study would also be shared and discussed with these players as an essential step in improving the quality and impact of Technical Assistance. However it should be noted that the study does not intend to evaluate the performance, outcomes or impact of any specific player but only collects perceptions of the users and providers of Technical Assistance and Support.
In keeping with the “Three Ones” principles, the study would examine the Global Fund grant within the context of the wider national programme even in the case of a sample country not having a coordinated national response or local ownership.
Sample countries
The sample countries selected for the study represent a range of countries across the world at different stages of the epidemic and levels of inputs and efforts in AIDS prevention, care, support and treatment. The role of UNAIDS, its Co-sponsors, bilateral and donor agencies, INGOs are also much different between these countries. Some of these sample countries are also covered by the ongoing 5-year evaluation of the Global Fund (SA2 – Effectiveness of the Global Fund Partner Environment) that also evaluates Technical support.
This study, unlike the Global fund 5-year evaluation, is not an evaluation but involves case studies, problem identification and search for solutions. The sample countries selected are:
Asia: India, Cambodia, and Vietnam
Africa: Zambia, Ethiopia, and Senegal
Latin American and Caribbean: Peru, Haiti
Eastern Europe: Ukraine
Methodology
The broad methodology for the study will be the ‘case study’ method. Information will be gathered through the following:
(a) Interviews & discussions with providers (both consultants, and donors who fund Technical Support to Global Fund Implementation) and users of Technical Assistance & Support
(b) review of records and documents
(c) self-administered questionnaires , and
(d) workshops.
Self administered tools, Qualitative discussion guides and information checklists will be used by the researchers. Certain cases of informal support provided that were critical or important will be documented. Should there be numerous instances of Technical Assistance(TA) and Technical Support(TS), purposive sampling of those TA/TS instances will be done, to ensure a broad coverage of the different types of support, different players, different regions within a country, and different stages of the programme cycle. It will primarily focus on all instances of TA & TS during GFATM grant implementation from 2007 onwards, from proposal development, technical reviews, grant negotiation, and Implementation (programmatic, financial, governance, M & E, reporting).
Reference Period
The data collected for the study would relate to the period since proposal preparation for the 2007 Round the Global Fund Grants or earlier rounds where there was no Grant for the 2007 Round of GF grants.

1. Objectives, Scope, Methodology & Outputs
1.1 Objectives
1.1.1 Assessment
To collect information, document and assess the different instances of Technical Assistance provided to national AIDS programmes by different providers since 2007 to support Global Fund grant implementation in a sample of countries across the world.
1.1.2 Demand
To study and document the technical support needs of implementation of national AIDS programmes implementing Global Fund grants at different stages of implementation and at different levels and the extent to which such needs are evident and felt by the programme managements in each sample country.
1.1.3 Supply
To study and document the availability, accessibility, pricing and distribution of Technical Support for Global Fund Grant implementation from different sources in each sample country.
1.1.4 Recommendations
To evolve and recommend practical solutions to problems, deficiencies and gaps identified in demand, supply, structures, programmes and policies relating to Technical Assistance and Technical Support to the Global Fund Grant Implementation.
1.2 Scope
1.2.1 Coverage
While the study is initiated by UNAIDS, the study would examine Technical Assistance and Support to Global Fund grants provided by all players to the National AIDS programmes in the sample countries. The outputs of the study would also be shared and discussed with these players as an essential step in improving the quality and impact of Technical Assistance. However it should be noted that the study does not intend to evaluate the performance, outcomes or impact of any specific player but only collects perceptions of the users and providers of Technical Assistance and Support.
In keeping with the “Three Ones” principles, the study would examine the Global Fund grant within the context of the wider national programme even in the case of a sample country not having a coordinated national response or local ownership.
1.2.2 Sample countries
The sample countries selected for the study represent a range of countries across the world at different stages of the epidemic and levels of inputs and efforts in AIDS prevention, care, support and treatment. The role of UNAIDS, its Co-sponsors, bilateral and donor agencies, INGOs are also much different between these countries. Some of these sample countries are also covered by the ongoing 5-year evaluation of the Global Fund (SA2 – Effectiveness of the Global Fund Partner Environment) that also evaluates Technical support.
This study, unlike the Global fund 5-year evaluation, is not an evaluation but involves case studies, problem identification and search for solutions.
The sample countries selected are:
Asia: India, Cambodia, and Vietnam
Africa: Zambia, Ethiopia, and Senegal
Latin American and Caribbean: Peru, Haiti
Eastern Europe:Ukraine
These 9 sample countries have been selected on the basis of:
● Strong UN presence
● Significant bi-lateral presence
● Recipients of Global Fund HIV grant
● Mixture of three African, three Asian, one Eastern European, and one Latin American and one Caribbean country.
● With reference to the on-going Global Fund 16 country study of TA Provision (part of the Five Year Evaluation)
1.2.3 Reference period
The data collected for the study would normally relate to the period since 2007 or since Technical Assistance was used in a sample country for the 2007 round of the Global Fund proposal preparations. In case sufficient instances of Technical support are not found, the reference period would change for certain parameters to the previous year or to the period since efforts began in a sample country for HIV prevention, care, support and treatment efforts.
1.2.4 Working Definitions
Technical Assistance & Technical Support
Technical Assistance involves provision of money, materials or human resources to enable new knowledge, building skills and capacities, providing services or providing new technology to those who may not have it.
Technical Support refers to the final delivery of required knowledge, training or services.
To illustrate, a donor may provide Technical Assistance by identifying and paying for the services of a consultant or a team of consultants along with all the associated costs.
Technical support refers to the act of providing services by the consultant, irrespective of who funds the activity.
Instances
A single Technical Assistance grant or a group of Technical Assistance Activities for a defined set of outputs or a single output such as Global fund proposal will be termed Technical Assistance Instance.

A single Support effort by a consultant or a firm with a defined set of outputs or a single output such as a report, training or proposal will be termed Technical Support Instance.
Informal instances are also included in this definition where the Assistance or Support are critical, path breaking or important.

Long & Short Term Consultancy
Different Technical Assistance providers or users define the terms ‘long’ and ‘short’ term differently. Some consider any consultancy for periods below 3 months as short term. This study would assess instances of Technical Support in terms of adequacy, relevance, effectiveness and efficiency without naming it as “long” or “short” term. The study would assess if Technical support has been provided as long as needed or till capacities have been built locally.

Other Definitions
Any other term with scope for differing interpretations would be defined before use in the tools or reports of the study.

1.3 Broad Methodology

The broad methodology for the study would be the case study method. Data collection for the case study would employ a range of methods including desk research, key informant interviews of different stakeholders, qualitative research methods such as in-depth interviews , self-assessment questionnaires and workshops.
1.4 Tools
The following tools and Manuals are designed for the study.
1. Self Assessment questionnaires for UNAIDS & Cosponsors
2. Online Workbook with space for Content Analysis heads
3. Interview Guide for Technical Support User organizations
4. Interview Guide for Technical Assistance / Technical Support Providers
5. Interview Guide for other Stakeholders
6. Study Manual
1.5 Outputs
1.5.1 Documentation (10)

Documentation for each country and at the global level providing the study process documentation, data, analysis and findings of the study efforts.
1.5.2 Country Case Studies (9)
10 page summaries of the study findings in each sample country.
1.5.3 Global Report (2 stages)
Interim Global Report summarizing findings from the field work stage in 9 countries by November 24.
Final Global Report Summarized integrated findings from the Country Case Studies and from the interactions with global players with recommendations.

2. Study Approach

The Study has 4 Objectives in studying TA & TS for Global Fund Grant Implementation
A. Assessment B. Demand C. Supply D. Recommendations
2.1 Assessment Objective
To collect information, document and assess the different instances of Technical Assistance provided to national AIDS programmes by different providers since 2007 to support Global Fund grant implementation in a sample of countries across the world.
Based on interviews with the providers of technical assistance to the Global Fund Grant implementation, details of instances of technical assistance within the reference period will be collected along with reports and documents that the providers are willing to share. An instance of Technical Assistance could involve a single instance of Technical support or a series of Technical Support instances. Each such instance of support will be listed along with data on nature of support, identity of the support provider/ consultant, the inputs (costs and duration of support.), processes, outputs and outcomes. If there have been very few such instances, the reference period will be extended to obtain more instances. If there are very few formal instances, informal cases of Technical support will be documented from narrations. Even where there are a large number of formal Technical Support instances, there could be specific informal instances that could have provided important or critical support to the national programme. Such informal Support cases would also be documented. If the total number of instances are larger than expected and cannot be covered in the time budgeted for this study, purposive sampling of the instances would be done ensuring coverage of different types of support, different players, different regions within the country and different stages of the programme cycle.
The reports and documents will be studied during desk research to draw insights on the parameters for Assessment. Interviews with users of the technical support will be conducted.

The steps in meeting the Assessment objective are:
a) Interviews with providers of Technical Assistance and collection of records/documents
b) Interviews with providers of actual Technical Support (if different from provider of Technical Assistance- as in donor-consultant)
c) Desk review and preparation
d) Interviews with users
e) Generating Outputs that will serve as inputs to other objectives
The study will assess the following:
A. Adequacy and Relevance
The UN Division of Labour is expected to increase the effectiveness and efficiency of the UN response. Based on self-assessment by the Local offices of UNAIDS and Cosponsors, and based on the interviews with the Technical Support users, the study would assess the adequacy and relevance of the Technical support to the mandate (in addition to relevance to the needs of the client). If problems such as “mandate creep” and inability to adhere to the Division of Labour are reported, these would be analysed with the help of follow up interviews with the relevant offices. The other providers of Technical support such as bilateral agencies have their own mandates and Technical Support by these providers will also be assessed for relevance to needs and to mandates.
B. Quality of Support
Quality of Support (QoS) for the purpose of this study would mean the extent to which support delivery meets user expectations. A full evaluation against standards and benchmarks is beyond the scope of this study.
C. Access to support
The degree to which Technical Support is unrestricted by geographic, economic, social, organizational, or linguistic barriers indicates Access to Support. Here again, this study has to restrict measurement of Access based on the client’s opinion and level of comfort with the access to technical support that is available. Besides the client’s comfort level, the study will attempt to identify if there are any geographic, economic, social, organizational, or linguistic barriers through anecdotal information from providers and clients.
D. Effectiveness of Support
The degree to which desired results (outcomes) of TS are achieved will be explored. For some TS efforts the outcomes are somewhat measurable such as TS provided for Global Fund proposals resulting in grant awarded or Phase II negotiations leading to revised grants. For some other TS efforts (such as implementation support) the measurement of Outcomes becomes complicated and Outcomes are shared between several efforts only one of them being Technical Support. This study will attempt a qualitative description of results of Technical Support that indicate Effectiveness of Support.
E. Efficiency of support delivery
Efficiency is measured by the ratio of the outputs of TS to the associated costs of providing TS. While outcomes pose a challenge in attribution, (hence measurement) Outputs would be clear. Cases where TS Succeeds or fails to deliver planned outputs can be easily detected. Successes and Failures due to “timeliness” or lack of it would also be counted.
F. Interpersonal relations
Interpersonal relations are characterised by trust, respect, confidentiality, courtesy, responsiveness, empathy, effective listening, and communication between providers themselves and between providers and clients. Anecdotal evidence about success and failure cases attributed to Interpersonal/ Inter-organizational relationships would be collected.
G. Continuity of Support
Delivery of Support by the same provider throughout the course of Support (when appropriate) and appropriate and timely coordination and communication between providers and building of local capacities by International providers so that the local providers take over the delivery responsibilities would indicate Continuity of service. Where Technical Support has been provided in bits and pieces without Continuity, irrespective of the duration of support, the support would be deficient.
2.2 Demand & Supply Objective
To study and document the technical support needs of implementation of national AIDS programmes implementing Global Fund grants at different stages of implementation and at different levels and the extent to which such needs are evident and felt by the programme managements in each sample country.

To study and document the availability, accessibility, pricing and distribution of Technical Support for Global Fund Grant implementation from different sources in each sample country.

The stages of implementation of a Global Fund grant programme for the purpose of this study would be;
a) Proposal Development / Revision for Phase II
b) Grant Negotiations
c) Phases One and Two of Implementation (including Programme Implementation; Financial Management; Procurement and Supply Management; Monitoring and Evaluation; Governance; Prevention, Treatment and Care)
d) Reporting

The Demand and Supply Analysis will be based on the following steps;
a) In-depth Discussions (one to one) with different user stakeholders (CCM, NAC, Regional Implementers, NGOs/ Other project implementers / grantees) on the need for different types of technical support at each stage of implementation. Opinions on the Demand and Supply situation for Technical Support in the country, past experiences with international and local consultants, levels of satisfaction etc. will also be collected.
b) Self Administered questionnaires from UNAIDS & co-sponsors at country level on Demand for Technical Support, Supply of Funds and Technical support, cost of consultants, procedures for hiring http://start.ubuntu.com/8.04/consultants in the country including need for external consultants.
c) Interviews with bilateral, other donors and stakeholders on their perception of demand and supply for each type of Technical Support, their processes for selecting, hiring and paying consultants.
d) A sample of consultants and key informants, purposively sampled to cover information gaps in market study will be interviewed.
2.3 Recommendations
The Recommendations objective is to evolve and recommend practical solutions to problems, deficiencies and gaps identified in demand, supply, structures, programmes and policies relating to Technical Assistance and Technical Support to the Global Fund Grant.
Throughout the research, the study team members would be asked to separately record problems, deficiencies and gaps that are reported by different respondents as well as ideas that are promising to develop into solutions.
Online brainstorming would also be carried out through different online forums and email discussions throughout the study period.

3. Broad Study Activities

The following are the broad study activities recommended at the National level.
Study Activities
Contracting
Understanding the study
Preparatory Tasks
Initial Respondents List
Field Work & Analysis
Online updating / interaction
Stakeholder Feedback
Draft Report
Final Report

Each of these activities are described below:
3.1 Contracting Formalities
All the study team members are on board. and contracted. The study team have been recruited in the 9 countries through Technical Support facilities and other partners.
3.2 Understanding the study
The online discussion group is a great place to start. (restricted to study team)
You have received an invitation to the discussion forum. Login using a gmail.com id (any other id will allow you to receive emails from the discussion but you will miss out on other documents).
There is a small collection of “files”.Download each one and study them carefully. Do not hesitate to ask questions in the group about what to start reading.
There is a set of “Pages” such as this page relating to the study.
There are links to online docs emailed to you separately where many people can edit the same document/ spreadsheet or presentation simultaneously.
Above all there are people from all over the world happy to interact with you. One of these people is the lead consultant.
There will be many doubts, questions and concerns you may have in the initial period of reading. The best way to deal with these is to ask / type. Ask the lead consultant to call you on phone or post a question to everyone in the group.
Besides these there are useful documents on the site http://theglobalfund.org including those relevant to your country. There are other sites and documents across the cyber world. If you discover a great document or site, do share the information with the others.
3.3 Preparatory Tasks
The study preparation Tasks other than reading the references includes:
1. Deciding Study reference period for your country
2. Listing study Respondents
3. Listing TA/TS Instances
4. Grouping Contacts
5. Sampling Instances & related Contacts
3.3.1 Study Reference Period
As required by the study, we are looking at UN support to Global Fund Grant Implementation.
From the website of the Global fund and from your own knowledge you will know which is the last GFTAM round of grant in your country.
Consider only the GFTAM Grant for HIV related interventions.
For the recent round consider all the stages of the grant from proposal development to Reporting stage.
If certain stages have not been reached for the most recent round, extend your study to the previous round activities for the specific stage.
We are not concerned with earlier rounds if there are enough instances in the recent round. This is because the study is about current situation on TA/TS provision and we will discuss the history only if it helps the objectives of the study.
3.3.2 Listing Study Respondents

Having understood the study and knowing the response to HIV in your country well you will be able to quickly list many potential Respondents in the country for the study. List the organizations first and later add the key people in that organization you would like to meet for different kinds of information. Where you don’t know, you will know people who know. Phone them, meet them, prepare the list. If during the study you come across more such Respondents, keep adding them even if you don’t plan to meet them for this study. Please remember, we are now looking at respondents related only to the Global Fund Grant implementation.
3.3.3 Grouping Respondents
The following types of Respondents are needed
1. UN System Respondents
(for interviews as well as self administered interviews.)
2. CCMs and NACs
(CCM- Country Coordination Mechanism, NAC- National AIDS Councils Example NACO in India)
3. PRs and big SRs
(PR- Principal Recipient , SR Sub Recipients of Global Fund Grants for the grant round being covered)
3. Professional TS providers to Global Fund implementation
(Non profit, commercial providers of TS during the study reference period)
4. Other large providers of TS to Global Fund Grant implementers:
(Donor / Bilateral agencies/Government agencies/ HIV specific groups/organizations, Independent Agencies involved in the reference period )
3.3.4 Listing TA/TS Instances

TA/TS Instances will be found at different stages of the Global Fund Grant Implementation such as :
a) Proposal Development / Revision for Phase II
b) Grant Negotiations
c) Phases One and Two of Implementation (including Programme Implementation; Financial Management; Procurement and Supply Management; Monitoring and Evaluation; Governance; Prevention, Treatment and Care)
d) Reporting
All instances will be listed in the database.
3.3.5 Sampling Instances
There is no limit on the number of instances you can cover within the study reference period. Do a purposive selection of instances across grant Implementation stages, across major TA/TS providers, across TA/TS users and across types of TA/TS. Consider the major providers & users and the instances of TA/TS they are involved in. Also select the odd TA/TS instances where there appears to be “mandate creep” or other issues.

The selection of instances for your case study is linked to the selection of respondents such as TA/TS providers and users for your study. Try to cover all the instances of TA/TS from UNAIDS and its co-sponsors. Identify instances of TA/TS to Global Fund Grant Implementation – from Bilateral agencies and key donors in your country. Cover different types of TA/TS instances. Do not breach the reference period till you specifically extend it.

The study is designed to cover 20-30 respondents in-depth in each country. Some countries may need more respondents and some less. Involve the Lead consultant in finalizing the number of respondents you will cover.
3.4 Initial Respondents list

You can start with the UN , CCM, NAC and major implementers of GFTAM grants as the starting point to conduct the interviews. After you are clear about the Instances list, you can examine the other Respondents.
3.5 Field Work & Analysis
3.5.1 Field Work
The field work tasks include:
1. Following up on self administered questionnaires (UN)
2. Fixing appointments and conducting interviews
3. Collecting data/ documents and entering data
4. Maintaining informal contact with the study respondents
5. Recoding interviews, observations (field diary) and updating the online workbook
6. Posting interesting / illustrative field experiences and learnings for the benefit of other country researchers, for later research and for posterity.

The interviews involve collecting two broad types of study inputs:
A. Facts and data
The evidence for facts and data are documents, collect copies of all relevant documents that contain such evidence. The tool for collecting facts are check lists. Prepare for each interview by revising the checklist of facts that you will need from the respondent.
B. Qualitative inferences
Since all researchers are well experienced, the preparatory tasks do not include any training in qualitative research methods. However a clear understanding of the do’s and don’ts of qualitative research is needed. For example unlike a quantitative survey, a qualitative researcher does not carry a questionnaire but carries only a discussion guide. The discussion guide itself is a flexible tool and changes focus as the study proceeds. It is a good practice to revise the discussion guide before / after every interview.
Understand interview questioning methods, ask probing questions but do not prompt the respondent, avoid leading questions and minimize bias. These are key qualities of a good qualitative researcher.

The qualitative interviews are recorded on voice recorders for later analysis. Acquire a good digital voice recorder that stores several hours of data. Ensure that the recorder is compatible with your computer. The recordings are important evidence for the study and are needed along with the documentation of the study. You may upload the recorded interviews towards the end of the study. Please note : Informed consent is required for all respondents , for being interviewed and for being recorded but the consent can be obtained orally. Provide a copy of the brief study note to the respondent while asking for appointments. Do switch off the recording when the respondent appears uncomfortable. Supplement the recordings with your own notes and observations as a field diary.
3.5.2 Analysis
The facts and data are recorded as lists and databases and analyzed as summary statistics and tables.

Qualitative Research -content analysis conventionally involves, recording, transcribing, coding, grouping sorting, re-sorting and drawing inferences. The recorded interviews are usually transcribed verbatim and then the content is analyzed. If you are the type of researcher who is more comfortable drawing findings by listening to the recorded interviews, go ahead.

If you are comfortable with content Analysis from Verbatim transcripts then you have to hire a local transcriber for your interview TA/TSrecordings and seek a budget for the transcriber. The transcripts along with the recordings will be part of the study documentation.

If you post your findings every day on the online work-book and field diary, you will have less work of analysis. you would just have to cross check your inferences, compare and revise. What you post every day can always be edited and changed later.
3.6 Online interactions
The key difference between this study and most other research studies are the online interactions. The online workbook provides space for summarizing the qualitative research findings and sorting them. The data provided by the National consultants online will be used to quickly put together a preliminary Global Report. Without online feedback of field work every day, it will not be possible to complete the first phase of the report – Interim Global Report.

Whenever you have questions such as if you find that the structure of the workbook needs some changes do ask the Lead consultant to do so and expect a quick response. If you don’t get a response on it please email a reminder or phone, message online. The Lead Consultant can be reached on Telephone or Text Chat, Voice Chat or Video Chat 15 hours a day covering all study time zones on all days of the week till November 24.
Phone Numbers: (+91) 44 26512584 Mobile: (+91) 9444409954
Text chat/Voice Chat
google talk : crguna@gmail.com ,
yahoo : indeman ,
msn : crguna@hotmail.com
skype: crgunasekar
Text/Voice/Video chat
skype: crgunasekar

The workbook and field diary for each National Consultant are created. Please ask the Lead consultant for a link. If you are having difficulties accessing an internet connection, please hire suitable facilities after informing the contracting agency about the costs.

The discussion group is another forum where you can discuss almost anything related to the study and otherwise. There are other online tasks such as logging your daily attendance sheet, supporting the public discussion topics by ensuring participation from different stakeholders in your country.
3.7 Stakeholder Feedback.
Feedback can be obtained from different stakeholders by posting emerging findings during the last 3 weeks of the study. Meanwhile the Lead Consultant would open a discussion forum on the study issues and invite Global and local stakeholders to participate before the findings emerge and warm up the discussions on relevant topics. Ensuring participation in the online discussion forum is not easy. Your involvement in ‘marketing’ the forum to stakeholders in your country is the key. Innovative ideas for stakeholders participation is encouraged. We will not have some traditional ways of stakeholder feedback such as seminars and workshops during the study. If sufficient stakeholder feedback is not forthcoming you will have the additional burden of obtaining individual feedback.
3.8 Draft Report
The Global Report will be prepared in 2 stages. An interim Report from Online interactions with team members and uploaded information. The final report summarizing the national reports.
The online workbook will provide a skeleton of findings for your draft report. Start drafting the report as soon as you can and it does not matter if the report changes considerably as you go along. The size of the final output from the study is around 10 pages for your country report.
3.9 Final Report
Most of the work at this stage involves answering questions, pointing out the strength of the evidence for specific findings or rewriting portions of the report.


Study on Technical Assistance and Technical Support to Global Fund Grant Implementation at Country Level – Study Manual

Advertisements