How to Onboard Grant Reviewers Effectively: A Practical Guide for Funders

A well-run grant round can fall apart at the review stage. Even experienced assessors score inconsistently when they receive vague criteria, unfamiliar tools or no calibration exercise. The result is scoring spread that obscures real quality differences between applications, slower decision cycles, and frustrated panel members who are less likely to volunteer next time.

The problem is growing. The Association of Charitable Foundations reported that application volumes to UK foundations rose by 50 to 60 per cent in recent years, with some members seeing increases of 100 to 400 per cent (ACF, Foundations in Focus, 2025). More applications mean more reviewers, more coordination and more risk of inconsistency. Yet most funders still rely on emailed PDFs, ad hoc briefings and a hope that experienced professionals will figure it out.

Research confirms that training makes a measurable difference. A study published in PLOS ONE found that a brief calibration exercise raised inter-rater reliability from 0.61 to 0.89 among grant reviewers, with improvements observed for both novice and experienced panellists (Sattler et al., 2015). The evidence is clear: structured onboarding is not bureaucratic overhead but a direct investment in fairer, faster decisions.

This guide covers the full onboarding cycle, from preparation weeks before the round opens through to post-round learning. It is written for grants managers, programme officers and anyone who coordinates external or internal assessors.

Why does reviewer onboarding matter so much?

Reviewer onboarding directly determines the quality and fairness of funding decisions. Without it, panels produce scores that reflect individual interpretation rather than a shared standard. When agreement between reviewers falls below 80 per cent, more than a fifth of the data feeding funding decisions is effectively erroneous (McHugh, 2012).

The stakes are high. Over 14,000 UK grantmakers distributed more than £8.2 billion through foundations alone in 2023-24 (UKGrantmaking, 2024). Each scoring inconsistency increases the chance that a strong application is declined or a weaker one funded. For smaller funders running one or two rounds per year, a single poorly calibrated panel can undermine a whole programme cycle.

Poor onboarding also costs reviewer time. Panel members who lack clear guidance spend longer per application seeking clarification, re-reading criteria or second-guessing their own scores. Multiply that across a panel of eight reviewers assessing 200 applications and the wasted hours become substantial.

Good onboarding, by contrast, produces three measurable outcomes: higher scoring consistency, faster turnaround, and better reviewer retention. Assessors who feel prepared and supported are significantly more likely to return for the next round.

What should a reviewer briefing pack contain?

A briefing pack is the single most important document in your onboarding process. It should answer every question a reviewer might ask before they open their first application. At a minimum, it needs five components.

Fund objectives and priorities. Explain what the programme is trying to achieve, who it is for, and what a successful project looks like. Reviewers who understand the strategic context make better judgements about fit and feasibility.

Eligibility criteria. List what makes an applicant eligible or ineligible. This stops reviewers from spending time assessing applications that should have been filtered at triage. For guidance on structuring these clearly, see How to Write Clear Grant Criteria.

Scoring rubric with descriptors. Define each score on your scale. A five-point scale should have a plain-English description for every level, ideally with an example of what a response at that level looks like. Vague labels such as "good" or "satisfactory" invite inconsistent interpretation.

Conflict-of-interest policy. The Charity Commission's guidance (CC29) requires trustees and those involved in grant decisions to declare and manage conflicts. Your pack should explain how to declare a conflict, what happens when one is identified, and how the panel will handle split decisions. Further detail is available in Managing Conflict of Interest in Grants.

Practical logistics. Cover deadlines, the number of applications each reviewer will see, how to access the review platform, and who to contact with questions.

How do you run a calibration session?

Calibration is the exercise where all reviewers independently score the same sample applications, then compare and discuss their scores. It is the single most effective way to align interpretation before live scoring begins.

Select two to three sample applications. Choose one strong, one borderline and one weak example. If possible, use applications from a previous round rather than fabricating samples, as real submissions contain the ambiguity and nuance that reviewers will encounter in practice.

Score independently first. Each reviewer scores the samples without discussion. This reveals natural variation before any anchoring occurs.

Facilitate a structured discussion. Walk through each criterion in turn. Where scores diverge, ask reviewers to explain their reasoning. The goal is not to force agreement but to surface different interpretations of the rubric and resolve them. Update the rubric wording if the discussion reveals genuine ambiguity.

Record the outcomes. Document any rubric clarifications and share the updated version before the round opens. This creates a reference point reviewers can return to when they encounter edge cases.

A calibration session typically takes 45 to 90 minutes depending on the complexity of your fund. The PLOS ONE study on grant peer review found that even a brief training intervention significantly improved scoring accuracy and the amount of time reviewers spent reading criteria (Sattler et al., 2015). The return on a short calibration session is substantial.

What tools and workspace should reviewers have?

The review environment shapes reviewer behaviour. Scattered documents, unclear navigation and manual score submission all increase the chance of errors and reviewer fatigue.

At minimum, reviewers need a system that provides each assigned application alongside the scoring criteria, collects structured scores and comments in one place, and tracks completion so managers know who is behind schedule. A shared Q&A channel or documented clarifications log prevents the same question being answered multiple times over email.

Some funders still circulate applications as email attachments with a spreadsheet scorecard. This approach breaks down quickly as volume grows. A late-2024 survey of trust fundraisers found that average success rates had fallen to around 35.6 per cent (Hinchilla, 2024), implying that each funded application now sits alongside roughly two unfunded ones that still require full assessment. Manual processes cannot absorb that volume without compromising quality or timelines.

Tools like Plinth take this further by providing a purpose-built assessment workspace. Funders can invite external assessors by email, organise them into groups, and bulk-assign applications to individuals or panels. The platform supports structured scorecards alongside a simple comments mode, so the assessment format matches the complexity of the fund. Privacy settings let managers redact identifying applicant information for blind review, or hide specific application questions from external assessors to protect sensitive data. An assessor summary dashboard tracks each reviewer's assigned, completed and outstanding assessments in real time, with progress bars that make it immediately clear who needs a reminder.

Plinth also offers configurable AI assessment verbosity, allowing fund managers to control how much detail the platform's AI assistant provides to reviewers, from brief highlights to detailed analysis. For funds where independence of judgement is paramount, AI features can be disabled entirely for external assessors with a single toggle.

How should you assign applications to reviewers?

Application assignment is where logistical planning meets fairness. The two most common models are full-panel review, where every reviewer sees every application, and distributed review, where each application is seen by a subset of the panel.

Full-panel review works well for small rounds with fewer than 30 applications. Every panellist scores everything, which produces robust data for comparison and simplifies moderation. The downside is that it does not scale; a panel of eight reviewers each assessing 200 applications means 1,600 individual reviews.

Distributed review suits larger rounds. The standard approach is to assign two or three independent reviewers per application. Research on optimal reviewer numbers suggests that two independent reviewers provide a reasonable balance between reliability and efficiency, with additional reviewers added for higher-value or higher-risk applications (Pier et al., 2017).

When distributing, consider expertise matching, geographic or thematic relevance, and declared conflicts of interest. Random assignment within these constraints prevents unconscious clustering.

Plinth supports both models. Fund managers can assign all selected applications to chosen assessors in one step, or randomly distribute applications between assessors with a configurable number of applications per reviewer. Assessor groups allow thematic panels to operate independently within the same fund. For more on handling volume, see How to Manage Large Volumes of Applications.

How do you manage external and volunteer reviewers?

Many funders rely on external experts, community representatives or volunteer panellists alongside staff assessors. These reviewers bring valuable perspectives but often lack familiarity with the funder's systems, terminology and expectations.

External reviewers need everything that internal staff receive plus additional context. They may not know what your organisation's strategic priorities are, how previous rounds were run, or what the typical profile of a successful applicant looks like. A short introductory document or a 15-minute video call can bridge this gap.

Access management is equally important. External reviewers should see only the applications assigned to them, not the full pipeline. Where blind review is required, applicant-identifying information should be redacted before the reviewer gains access. The Charity Commission's guidance on conflicts of interest applies to all decision-makers, not just trustees, so external panellists must complete the same declarations as internal staff (Charity Commission, CC29).

Communication cadence matters. Send a welcome message when access is granted, a midpoint check-in when roughly half the deadline has passed, and a reminder three to five days before the scoring deadline. Keep a central log of all clarifications so that answers given to one reviewer are visible to all.

After the round, thank external reviewers promptly. Maintain a talent pool with notes on each reviewer's expertise, reliability and any feedback they gave about the process. Good reviewers are a scarce resource, and retention depends on the experience you provide. According to NCVO, people who volunteer give an average of 8 hours per month, valued at approximately 19 pounds per hour (NCVO, UK Civil Society Almanac, 2024). Respecting that time builds long-term relationships.

What does good scoring consistency look like?

Scoring consistency, often measured as inter-rater reliability, indicates how much agreement exists between independent reviewers assessing the same application. Perfect agreement is unrealistic and undesirable, as some divergence reflects legitimate differences in professional judgement. But wide divergence signals a problem with criteria, calibration or both.

A useful benchmark comes from research: an intraclass correlation coefficient (ICC) above 0.75 is generally considered good reliability, while values below 0.50 indicate poor reliability where nearly half the variance may be noise rather than signal (Koo and Li, 2016). The PLOS ONE grant review study achieved an ICC of 0.89 in the trained condition, demonstrating what is achievable with even modest calibration effort.

Practically, fund managers should monitor three indicators during a round:

1. Score spread per application. If one reviewer gives 4 out of 5 and another gives 1 out of 5 on the same application, the rubric or calibration needs revisiting.
2. Reviewer leniency or severity. Some reviewers consistently score higher or lower than the panel average. Identifying this pattern early allows you to address it through recalibration or statistical adjustment.
3. Completion rates. Reviewers who fall behind on deadlines may rush their remaining assessments, reducing quality. Tracking progress in real time allows timely intervention.

Plinth's assessor summary dashboard displays assigned, completed and incomplete counts for each reviewer alongside progress percentages. Fund managers can sort and filter by completion status, making it straightforward to identify who needs support. Where total scores are enabled, the platform calculates and displays per-assessor totals and averages so scoring patterns are visible at a glance.

What should happen after the review round?

Post-round activities are where most funders under-invest, yet they are the foundation for improving the next cycle. Three activities deserve attention.

Moderation and edge-case review. Identify applications where reviewer scores diverged significantly and convene a brief moderation discussion. This is not about overruling individual reviewers but about ensuring that borderline decisions are defensible. Document the rationale for every close call.

Feedback to applicants. Rejected applicants benefit from constructive feedback, and providing it strengthens your reputation and the quality of future applications. For guidance on structuring this, see Why Feedback Builds Better Funders. Where volume makes individual feedback impractical, provide thematic summaries of common strengths and weaknesses.

Process retrospective. Gather feedback from reviewers about the onboarding experience, the clarity of criteria, the usability of the platform and the manageability of their workload. Specific questions yield better data than open-ended requests for comment. Update your briefing pack, rubric and calibration samples based on what you learn.

Export your assessment data for analysis. Plinth allows fund managers to export all applications with assessment scores as a CSV, or as formatted Word documents for board papers. This data supports year-on-year comparison and helps identify systemic patterns in scoring.

Update your reviewer talent pool. Record who participated, their areas of expertise, whether they completed on time, and any feedback they offered. A well-maintained pool accelerates recruitment for the next round and reduces the onboarding burden for returning reviewers.

What are common onboarding mistakes to avoid?

Even well-intentioned funders make predictable errors when onboarding reviewers. Recognising these patterns helps you avoid them.

Assuming experience equals calibration. The PLOS ONE study found that experienced reviewers also benefited significantly from training. Familiarity with grantmaking does not guarantee alignment with your specific rubric and fund priorities.

Overloading reviewers. Assigning too many applications per reviewer leads to fatigue, declining quality in later assessments and higher dropout rates. A reasonable target depends on application complexity, but for detailed assessments most experts recommend no more than 15 to 20 applications per reviewer per round.

Skipping the conflict-of-interest step. It is tempting to trust that professionals will self-declare, but formal declarations protect both the reviewer and the funder. The Charity Commission is explicit that conflicts must be identified and managed proactively (Charity Commission, CC29).

Using undefined scoring scales. A scale of 1 to 5 means nothing without descriptors. If "3" is not defined, one reviewer may use it as a default for uncertainty while another reserves it for genuinely average applications. The result is noise disguised as data.

Providing the tool without a walkthrough. Even intuitive platforms benefit from a brief orientation. A five-minute screen recording showing how to log in, navigate to assigned applications, enter scores and submit can prevent a wave of support queries on day one.

Neglecting the post-round debrief. Every round generates learning. Funders who skip the retrospective repeat the same problems and gradually lose their best reviewers to better-organised programmes.

Frequently Asked Questions

How long should reviewer training take?

A calibration session of 45 to 90 minutes is usually sufficient, depending on the complexity of the fund. This should include independent scoring of two to three sample applications followed by a facilitated discussion of scoring differences. Supplementary materials such as a briefing pack and a short platform walkthrough can be provided asynchronously.

Should reviewers meet each other before the round?

A brief meeting or video call helps build shared understanding, particularly for new panels. For established panels with returning reviewers, an asynchronous calibration exercise with a written debrief can be equally effective. The key is that all reviewers discuss their interpretation of the scoring criteria before live assessment begins.

Can non-experts serve as grant reviewers?

Yes, provided they receive clear criteria and appropriate support. Community representatives and service users bring perspectives that technical experts may lack. Pair them with subject-matter specialists where possible, and ensure the rubric is written in plain language that does not assume sector-specific knowledge.

How many reviewers should assess each application?

Two independent reviewers per application is a common standard for most grant rounds. Higher-value or higher-risk applications may warrant three or more. Research suggests diminishing returns beyond three reviewers for routine assessments, but the right number depends on the stakes involved and the resources available.

What is blind review and when should we use it?

Blind review means reviewers assess applications without seeing the applicant's name, organisation or other identifying information. It is most valuable when there is a risk of unconscious bias, for example when the funder has existing relationships with applicants. It adds administrative overhead, so use it where the fairness benefit justifies the effort.

How do we handle a reviewer who consistently scores too high or too low?

First check whether the issue is calibration or criteria interpretation. A conversation referencing the rubric descriptors often resolves the problem. If the pattern persists, consider statistical adjustment of that reviewer's scores or additional calibration exercises. Avoid removing a reviewer mid-round unless there is a conduct concern.

What should we do if reviewers disagree significantly on an application?

Significant disagreement, typically a spread of more than two points on a five-point scale, should trigger a moderation discussion. Ask the diverging reviewers to explain their reasoning, then facilitate a panel conversation. Document the outcome and the rationale. Disagreement is not a failure; unresolved disagreement is.

How can technology help with reviewer onboarding?

Grant management platforms can automate application assignment, track reviewer progress, enforce conflict-of-interest declarations before access is granted, and provide structured scoring templates that reduce interpretation errors. Tools like Plinth additionally offer AI-assisted application summaries, configurable privacy settings for blind review, and real-time assessor dashboards. The platform has a free tier, making it accessible to smaller funders.

Recommended Next Pages

• How to Automate Reviewer Workflows — Reduce admin around reviews while keeping decisions with people.
• How to Write Clear Grant Criteria — Structure eligibility and assessment standards for applicants and reviewers.
• Managing Conflict of Interest in Grants — Identify, declare and manage conflicts across your grant programme.
• How to Manage Large Volumes of Applications — Scale assessments without increasing staff costs.
• How to Build Transparency into Grant Decisions — Make your decision-making process open and defensible.

Last updated: February 2026