Identifying what works

With more rigorous studies and higher quality evidence, more certainty about what works, for whom, and in what context can be achieved.

• Evidence-based programs are programs that have been rigorously tested and evaluated. They have been proven effective and translated into practical models that are suitable for adoption into local settings.
• Research-based programs are not necessarily evidence-based. They are programs containing research-based content, or programs guided by research, but they have not been proven effective and are not evidence-based.
• Best practice is a generally accepted method that is acknowledged across the sector. Best practice can be based on or informed by evidence, but typically has not undergone the same scientific evaluation to validate effectiveness. A major criticism of best practice is that the work required to establish that something is ‘best’ is rarely done and even ‘better’ practices, or ‘good’ practice will only hold true in certain contexts.

See our report What works? Learning from the literature (2015).

Evidence-based practice (EBP) follows five basic steps.

1. Identify the gap (what you want to know)

Formulating precisely what you want to know is the first step. A widely used approach to identifying the question is the ‘PICO’ approach.

• Population: What are the characteristics and/or conditions of the target group (e.g., self-represented individuals seeking legal help at Family law courts)?
• Intervention: What is the intervention or service delivery model that you are considering (e.g., duty lawyer services)?
• Comparison: What is the main alternative to the chosen intervention (e.g., different intervention, current intervention, no intervention)?
• Outcome: What is it you want your intervention to achieve, measure, or improve (e.g., reduce the impact of self-represented litigants on the workload of the Family Law Courts)?

2. Searching for the best available evidence

• How to plan your search: The answers to your PICO questions will enable you to create search terms to help narrow your internet search using libraries, databases, google search or other appropriate websites (see below for where you could search for evidence).
• What type of external evidence do you need?: There are specific study designs that are optimal for addressing different types of questions so having your question well defined is important. For example, knowing which service model achieves the intended outcomes more economically will be best addressed with a cost-effectiveness study.
• Synthesised evidence: Searching for synthesised evidence can save time as the work is already done for you. These can be in the form of systematic reviews, meta-analyses, or evidence-based guidelines. However, these may be harder to find for the legal assistance sector as the body of evidence is still emerging.
• Where should you search for evidence: Searching for evidence can be a challenging undertaking and knowing where to search will save you valuable time. Many specific fields of work have clearinghouses or focused databases to start your search but much of the work done in the legal assistance sector will be found at the organisations publication pages. For some suggested links see the below tab, where to search for evidence.

3. Critically analysing the evidence for its validity and usefulness

• Evidence from research can assist in determining whether an intervention or service delivery model is likely to work in your context. Critically analysing the evidence is necessary to assess the quality (relevance, validity, and trustworthiness). However, this may require significant research expertise to confidently make conclusions about the evidence. To critically analyse the evidence, several key questions of each study should be considered.
• Is the study relevant to your question? Does it examine a similar population to your target population, are the elements of the intervention/service model within your resources and skills, are the outcomes relevant?
• Is the evidence valid, and useful? This includes the research procedure, the sampling methods, data collection methods, and analysis. Is the methodology appropriate, does the research design support the research question and what are the limitations of the study? Limitations are crucial to examine as they can significantly influence the outcomes and introduce bias or systematic errors.
• Are the findings applicable? If the research is valid and useful you must ask yourself if the findings are robust and can you generalise the results to your specific circumstances? To answer these questions statistical analyses, outcomes, consistency with other known evidence, and generalizability must be considered.

4. Integrating the findings with sector expertise and client needs

• The evidence from literature alone is not enough. To apply evidence to local context, you should integrate the findings with your sector and professional expertise and the needs and perspectives of your target population.
• The importance of incorporating the user perspective in service design is well known, but the terminology used can be important in operationalising this approach.

See our resource where we discuss the nuances behind the terms and what people-centred justice means.

5. Evaluating performance or outcomes

• Implementing your evidence-based program requires appropriate monitoring and evaluation to ensure that the intervention is appropriate, works in your local context and to support any adjustments as needed.
• Evaluations must also be subjected to critical peer review, by appropriately qualified professionals.

See our research and policy page for a list of organisations that have relevant research to legal assistance and justice system services.