15 Bayesian Shrinkage: Ranking Managers and Teams Fairly

Welcome back

This is the chapter most directly behind this book’s existence. Every People Analytics team eventually gets asked for a manager league table — who has the most engaged team, the best retention, the fewest complaints. And every People Analytics team eventually discovers the same trap: the “best” and “worst” managers on a raw league table are usually the ones with the smallest teams, not the ones actually doing the best or worst job.

Chapter 14 introduced empirical Bayes as a fast, closed-form fix for this. This chapter fits the full Bayesian version — a multilevel model — so you get a defensible answer with proper uncertainty for every manager, not just a point estimate.

Important

If you take one technique from this book back to your desk, make it this one. It changes how you should present any manager or team comparison you’re ever asked to produce.

What you’ll be able to do by the end

Explain why a manager’s team size affects how much you should trust a metric calculated about their team
Fit a Bayesian multilevel model for a rate with an exposure (events per person managed)
Extract and visualise shrinkage for individual managers, not just for aggregate groups
Compare a shrunk ranking to a raw one, and to an existing performance rating
Present shrunk results to a business audience that wants a simple league table

15.1 Setup

library(tidyverse)
library(peopleanalyticsdata)
library(brms)
library(tidybayes)

theme_set(theme_minimal(base_size = 13))
set.seed(2026)

data("managers", package = "peopleanalyticsdata")

# Guard against missing values before modelling (Chapter 1 found gaps
# in a similar teaching dataset).
managers <- managers |>
  drop_na(transfers, group_size, employee_id, city, performance_group)

We’re back with managers (571 managers, one row each). Two columns do the work in this chapter:

transfers — the number of transfer requests filed by people in a manager’s group while they’ve been the manager. Read as a warning signal: a group asking to leave.
group_size — how many people the manager is responsible for. This is the exposure: the number of chances a transfer request had to happen.

15.2 The problem: a league table that misleads

15.2.1 The raw rate

The obvious metric is transfers per person managed:

managers <- managers |>
  mutate(raw_rate = transfers / group_size)

managers |>
  select(employee_id, city, group_size, transfers, raw_rate,
         performance_group) |>
  arrange(desc(raw_rate)) |>
  head(10)

   employee_id          city group_size transfers  raw_rate performance_group
1     0ba1e858       Toronto          5         2 0.4000000            Bottom
2     1bb4fc5e      New York         13         5 0.3846154            Middle
3     314bc11c      New York         13         5 0.3846154            Middle
4     aa3a9fb5       Toronto         13         5 0.3846154            Middle
5     c848bdb0       Toronto         13         5 0.3846154            Bottom
6     5a343b35      New York          9         3 0.3333333            Middle
7     9b3dc002       Toronto          9         3 0.3333333            Middle
8     1623d0c9 San Francisco          6         2 0.3333333            Bottom
9     72cd94a4       Toronto          6         2 0.3333333            Bottom
10    27644867       Orlando         10         3 0.3000000            Middle

ggplot(managers, aes(x = group_size, y = raw_rate)) +
  geom_point(alpha = 0.5, colour = "#122a52") +
  geom_hline(yintercept = sum(managers$transfers) / sum(managers$group_size),
             linetype = "dashed", colour = "#d32f2f") +
  scale_x_log10() +
  labs(
    title = "Raw transfer rate by manager",
    subtitle = "Dashed line = organisation-wide rate. Small teams scatter to extremes.",
    x = "Group size (log scale)", y = "Transfers per person managed"
  )

15.2.2 The five-report problem

Look at the top of that table, and at the scatter plot: the most alarming raw rates cluster among managers with the smallest teams.

Important

A manager with 3 reports and 1 transfer request has a raw rate of 33% — worse than almost anyone else in the company on paper. A manager with 80 reports and 5 transfer requests has a raw rate of just 6%. Is the first manager really eight times worse? Or did one person, for reasons that might have nothing to do with management quality, ask for a transfer?

This is exactly the same arithmetic trap as Chapter 8’s guest-review counts and Chapter 14’s city complaint rates — but now the “group” with too little data is an individual manager, which is precisely the population most PA teams are asked to rank.

15.3 Three ways to handle it (recap)

Complete pooling — assume every manager has the same true rate. Ignores real differences between managers.
No pooling — trust every manager’s raw rate exactly as calculated. Over-trusts the small teams shown above.
Partial pooling — each manager gets their own estimate, pulled toward the organisation-wide rate by an amount the data itself decides, based on group_size.

15.4 Fitting the model

This is count data with an exposure that varies hugely across managers — the right tool is a Poisson (or negative binomial) model with group_size entered as an offset, and a varying intercept per manager so the model can borrow strength across all 571 of them:

Another log-scale intercept, so — same habit as Chapters 9 and 12 — translate it before trusting it:

set.seed(13)

tibble(log_intercept = rnorm(4000, -2.5, 1)) |>
  mutate(rate = exp(log_intercept)) |>
  ggplot(aes(rate)) +
  geom_histogram(bins = 40, fill = "#122a52", alpha = 0.6) +
  labs(title = "What Normal(-2.5, 1) on the log scale implies",
       subtitle = "Simulated draws from the intercept prior, translated to a per-person transfer rate",
       x = "Implied baseline transfer rate per team member", y = "Simulated draws")

Note

Centred around 8% (exp(-2.5)), with plenty of room either side — a sensible starting point for a rate we haven’t looked at yet, and nothing like a confident claim about the true organisation-wide figure.

fit_shrink <- brm(
  transfers ~ 1 + (1 | employee_id) + offset(log(group_size)),
  data = managers, family = poisson(),
  prior = c(prior(normal(-2.5, 1), class = Intercept),
            prior(exponential(2), class = sd)),
  chains = 4, iter = 2000, seed = 13, refresh = 0
)

summary(fit_shrink)

 Family: poisson 
  Links: mu = log 
Formula: transfers ~ 1 + (1 | employee_id) + offset(log(group_size)) 
   Data: managers (Number of observations: 571) 
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Multilevel Hyperparameters:
~employee_id (Number of levels: 571) 
              Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept)     0.88      0.10     0.70     1.08 1.00      986     1912

Regression Coefficients:
          Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept    -3.06      0.09    -3.24    -2.89 1.00     1682     2440

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

Reading this formula

transfers ~ 1 + (1 | employee_id) + offset(log(group_size)) says: each manager’s expected transfer count is their group_size (the offset — not something we’re estimating, just accounting for) times a rate. That rate is the organisation-wide rate (1, the Intercept) plus a manager-specific adjustment drawn from a shared distribution ((1 | employee_id)) — exactly the varying-intercepts idea from Chapter 8, just applied one manager at a time instead of one property at a time.

A different lens: this is a Bayesian version of a model you may already know

If you’ve fitted a mixed-effects / random-effects model before — lme4::lmer() or glmer() in R, or their equivalents elsewhere — you’ve already used the frequentist cousin of this exact model. The (1 | employee_id) syntax is even identical. The classical version gives you a point estimate for each manager’s random effect (a BLUP, “best linear unbiased predictor”) and a standard error built on asymptotic approximations that get shaky for small groups — precisely the groups we care most about here. The Bayesian version gives every manager a full posterior distribution instead of a point-plus-error-bar, which turns out to matter exactly when a manager’s team is small enough that “asymptotic” assumptions stop being a safe bet. Same shrinkage mechanism either way — the difference is what you get to say about the uncertainty afterward.

Only one row per manager — does that break anything?

No. The model doesn’t need repeated observations for each manager to learn from them collectively — it needs many managers, each contributing one data point of varying precision (set by group_size). It’s the same logic as estimating 571 baseball players’ true batting averages from a single season each: no one player gives you much, but the pattern across all 571 tells the model how much any one manager’s raw rate should be trusted.

15.4.1 What the group-level parameter tells you

sd_estimate <- summary(fit_shrink)$random$employee_id
sd_estimate

               Estimate  Est.Error  l-95% CI u-95% CI     Rhat Bulk_ESS
sd(Intercept) 0.8801281 0.09520379 0.6989286 1.076519 1.001642 986.0813
              Tail_ESS
sd(Intercept) 1912.429

sd(Intercept) here is a genuine finding: it says how much managers’ true transfer rates actually vary, once you strip away the noise from different team sizes. A large value says management quality (or at least whatever drives transfer requests) varies a lot across the organisation; a small value says most of the raw league table’s spread was noise all along.

15.4.2 Diagnose, then check

Two steps of the workflow before any estimate leaves this model, and they answer different questions.

Did the sampler work? summary() above reports Rhat and ESS for every parameter. This model has a group-level sd estimated from many small groups, which is the geometry Chapter 11 warns about, so it is worth a deliberate look rather than a glance.

Can the model produce data like ours? A count outcome needs the bar version of the check, as in Chapter 12:

pp_check(fit_shrink, type = "bars", ndraws = 100) +
  labs(title = "Posterior predictive check",
       subtitle = "Bars = what the model predicts; points = what actually happened",
       x = "Transfer requests", y = "Number of managers")

The failure to watch for here is the one Chapter 12 named: if the observed counts in the right-hand tail sit above their bars, the Poisson is under-covering and the shrunk estimates will be too confident. The negbinomial() swap from Chapter 12 is the fix, and it changes the intervals rather than the ranking.

15.5 Extracting each manager’s shrunk rate

To compare fairly, we ask the model for each manager’s expected rate per single person managed — set group_size to 1 in new data so the offset drops out, while keeping each manager’s own fitted adjustment:

shrunk <- managers |>
  mutate(group_size = 1) |>
  add_epred_draws(fit_shrink, re_formula = NULL) |>
  mean_qi(.epred, .width = 0.95) |>
  rename(shrunk_rate = .epred, lower = .lower, upper = .upper)

managers_compare <- managers |>
  select(employee_id, city, group_size, transfers, raw_rate,
         performance_group) |>
  left_join(shrunk |> select(employee_id, shrunk_rate, lower, upper),
            by = "employee_id")

managers_compare |> arrange(desc(shrunk_rate)) |> head(10)

   employee_id     city group_size transfers  raw_rate performance_group
1     314bc11c New York         13         5 0.3846154            Middle
2     c848bdb0  Toronto         13         5 0.3846154            Bottom
3     aa3a9fb5  Toronto         13         5 0.3846154            Middle
4     1bb4fc5e New York         13         5 0.3846154            Middle
5     a18ecc4e  Orlando         17         5 0.2941176            Bottom
6     5a343b35 New York          9         3 0.3333333            Middle
7     9b3dc002  Toronto          9         3 0.3333333            Middle
8     64e3a1ba  Toronto         10         3 0.3000000            Bottom
9     27644867  Orlando         10         3 0.3000000            Middle
10    bb595127 New York         10         3 0.3000000            Middle
   shrunk_rate      lower     upper
1    0.2375579 0.07118788 0.5310225
2    0.2359146 0.07428771 0.5246773
3    0.2334234 0.07470567 0.4953736
4    0.2326504 0.07424577 0.5119368
5    0.1925484 0.06305713 0.4085184
6    0.1709058 0.04053752 0.4351173
7    0.1694683 0.04174846 0.4245694
8    0.1637459 0.03759464 0.4165596
9    0.1632371 0.03975856 0.4147589
10   0.1619377 0.03769217 0.4103106

15.5.1 See the shrinkage

ggplot(managers_compare, aes(x = group_size)) +
  geom_hline(yintercept = sum(managers$transfers) / sum(managers$group_size),
             linetype = "dashed", colour = "#d32f2f") +
  geom_segment(aes(xend = group_size, y = raw_rate, yend = shrunk_rate),
               colour = "grey70", alpha = 0.6) +
  geom_point(aes(y = raw_rate), colour = "grey50", alpha = 0.5, size = 1.2) +
  geom_point(aes(y = shrunk_rate), colour = "#122a52", alpha = 0.6, size = 1.2) +
  scale_x_log10() +
  coord_cartesian(ylim = c(0, 0.4)) +
  labs(
    title = "Shrinkage across all 571 managers",
    subtitle = "Grey = raw rate, navy = model estimate, red dashed = org-wide rate",
    x = "Group size (log scale)", y = "Transfers per person managed"
  )

Small-team managers move the furthest — often dramatically — while large-team managers barely move, because their own data is already trustworthy.

Important

The manager with 3 reports and a 33% raw rate is not “the worst manager in the company” once you shrink properly — their small-team estimate gets pulled hard toward the organisational average, because three observations tell you almost nothing on their own.

15.6 Does the shrunk ranking agree with the official rating?

managers also has a performance_group label (Bottom / Middle / Top performer) from an actual review process. A useful sanity check: where does the shrunk transfer-rate ranking disagree with that rating, and is disagreement more common for small-team managers?

managers_compare |>
  group_by(performance_group) |>
  summarise(
    n = n(),
    mean_raw = mean(raw_rate),
    mean_shrunk = mean(shrunk_rate),
    mean_group_size = mean(group_size)
  )

# A tibble: 3 × 5
  performance_group     n mean_raw mean_shrunk mean_group_size
  <fct>             <int>    <dbl>       <dbl>           <dbl>
1 Bottom              129   0.0821      0.0736            11.4
2 Middle              376   0.0663      0.0675            11.8
3 Top                  66   0.0434      0.0583            12.6

managers_compare |>
  mutate(gap = abs(raw_rate - shrunk_rate)) |>
  ggplot(aes(x = group_size, y = gap, colour = performance_group)) +
  geom_point(alpha = 0.6) +
  scale_x_log10() +
  labs(
    title = "Raw vs. shrunk gap, by team size and performance rating",
    subtitle = "The gap that shrinkage corrects is concentrated among small teams",
    x = "Group size (log scale)", y = "|raw rate − shrunk rate|",
    colour = "Performance rating"
  )

Try it yourself

Filter to managers where the raw rate would have put them in the “worst 10%” of the league table, but the shrunk rate does not. How many are there, and what does their performance_group say? This is precisely the group a naive league table would have unfairly flagged.

For the ML/DS crowd

This exact problem — stabilising an estimate for an entity with too little of its own data, by borrowing strength from everyone else’s — is also how production recommender systems handle new or low-activity users and items. A user with two ratings gets their predicted preferences shrunk toward the population average; one with two thousand ratings is trusted almost entirely on their own history. It’s the same partial-pooling logic as this chapter, usually fitted with matrix factorisation and a regularisation term rather than brms, but solving an identical problem: don’t let sparse data pretend to be confident.

15.7 Presenting this to a business audience that wants a league table anyway

Stakeholders often want exactly the artefact this chapter argues against: a ranked list. You don’t have to refuse — you have to change what the list shows.

The deeper fix is to change what the list is for. A ranking invites the question “who is worst?”; a decision invites “who should we act on, and is acting worth it?” Chapter 24 takes these same posteriors and answers the second question, including the uncomfortable part — how many of the managers you pick will turn out to have been the wrong choice.

Three practical fixes for the deck

Show the credible interval, not just a point estimate. A manager whose 95% interval overlaps ten other managers’ intervals is not meaningfully different from them — say so, rather than presenting rank 14 as distinct from rank 24.
Report team size alongside the rank. “Rank 3 of 571 (team of 6)” reads very differently from “Rank 3 of 571 (team of 84)” — let the audience see the input to their own scepticism.
Group into tiers, not individual ranks. “Clearly above organisational average,” “not distinguishable from average,” “clearly below average” — based on whether the credible interval excludes the organisation-wide rate — resists the false precision of rank 47 vs. rank 48.

managers_compare |>
  mutate(
    org_rate = sum(managers$transfers) / sum(managers$group_size),
    tier = case_when(
      lower > org_rate ~ "Clearly above average",
      upper < org_rate ~ "Clearly below average",
      TRUE ~ "Not distinguishable from average"
    )
  ) |>
  count(tier)

                              tier   n
1            Clearly above average   4
2 Not distinguishable from average 567

This single table is usually the most useful thing you can hand a senior stakeholder: most managers, once you’re honest about uncertainty, are not distinguishable from average — and that’s a finding, not a disappointment.

On the job

Why this matters day to day

Any time you’re asked to rank people-metrics across managers, teams, stores, or offices of different sizes — attrition, engagement, complaints, absence, sales per head — this chapter’s approach applies directly. Publishing the raw league table is the single most common statistical mistake in People Analytics reporting, and it’s an easy one to fix once you know to look for it: check whether the extremes cluster among the smallest groups. If they do, shrink before you share.

Summary

Today you learned

Raw per-manager rates are least trustworthy exactly where team size is smallest — and those are usually the ones that end up at the extremes of a league table.
A Poisson model with group_size as an offset and a varying intercept per manager lets you partially pool rates across an entire population of managers, even with one row per manager.
Shrinkage pulls small-team managers’ rates toward the organisation-wide average, by an amount the model — not you — decides.
Comparing shrunk estimates to an existing performance rating is a useful sanity check, and often reveals that small-team managers were being mis-ranked.
You can still hand over a league table — just report uncertainty and team size alongside rank, or replace individual ranks with tiers.

Next chapter

Hierarchical models for pay and variance — we take the same partial-pooling machinery and point it at a different, equally sensitive question: how much pay variation actually sits at the level, function, or location layer, versus being individual noise?

# Bayesian Shrinkage: Ranking Managers and Teams Fairly ## Welcome back {.unnumbered} This is the chapter most directly behind this book's existence. Every People Analytics team eventually gets asked for a manager league table — who has the most engaged team, the best retention, the fewest complaints. And every People Analytics team eventually discovers the same trap: **the "best" and "worst" managers on a raw league table are usually the ones with the smallest teams**, not the ones actually doing the best or worst job. Chapter 14 introduced empirical Bayes as a fast, closed-form fix for this. This chapter fits the full Bayesian version — a **multilevel model** — so you get a defensible answer with proper uncertainty for every manager, not just a point estimate. ::: callout-important If you take one technique from this book back to your desk, make it this one. It changes how you should present *any* manager or team comparison you're ever asked to produce. ::: ## What you'll be able to do by the end {.unnumbered} 1. Explain why a manager's team size affects how much you should trust a metric calculated about their team 2. Fit a Bayesian multilevel model for a **rate with an exposure** (events per person managed) 3. Extract and visualise **shrinkage** for individual managers, not just for aggregate groups 4. Compare a shrunk ranking to a raw one, and to an existing performance rating 5. Present shrunk results to a business audience that wants a simple league table ------------------------------------------------------------------------ ## Setup ```{r setup} library(tidyverse) library(peopleanalyticsdata) library(brms) library(tidybayes) theme_set(theme_minimal(base_size = 13)) set.seed(2026) data("managers", package = "peopleanalyticsdata") # Guard against missing values before modelling (Chapter 1 found gaps # in a similar teaching dataset). managers <- managers |> drop_na(transfers, group_size, employee_id, city, performance_group) ``` We're back with `managers` (571 managers, one row each). Two columns do the work in this chapter: - `transfers` — the number of transfer requests filed by people in a manager's group while they've been the manager. Read as a warning signal: a group asking to leave. - `group_size` — how many people the manager is responsible for. This is the **exposure**: the number of chances a transfer request had to happen. ------------------------------------------------------------------------ ## The problem: a league table that misleads ### The raw rate The obvious metric is transfers **per person managed**: ```{r raw-rate} managers <- managers |> mutate(raw_rate = transfers / group_size) managers |> select(employee_id, city, group_size, transfers, raw_rate, performance_group) |> arrange(desc(raw_rate)) |> head(10) ``` ```{r raw-rate-plot} ggplot(managers, aes(x = group_size, y = raw_rate)) + geom_point(alpha = 0.5, colour = "#122a52") + geom_hline(yintercept = sum(managers$transfers) / sum(managers$group_size), linetype = "dashed", colour = "#d32f2f") + scale_x_log10() + labs( title = "Raw transfer rate by manager", subtitle = "Dashed line = organisation-wide rate. Small teams scatter to extremes.", x = "Group size (log scale)", y = "Transfers per person managed" ) ``` ### The five-report problem Look at the top of that table, and at the scatter plot: the most alarming raw rates cluster among managers with the **smallest teams**. ::: callout-important A manager with 3 reports and 1 transfer request has a raw rate of **33%** — worse than almost anyone else in the company on paper. A manager with 80 reports and 5 transfer requests has a raw rate of just **6%**. Is the first manager really eight times worse? Or did one person, for reasons that might have nothing to do with management quality, ask for a transfer? ::: This is exactly the same arithmetic trap as Chapter 8's guest-review counts and Chapter 14's city complaint rates — but now the "group" with too little data is an *individual manager*, which is precisely the population most PA teams are asked to rank. ------------------------------------------------------------------------ ## Three ways to handle it (recap) - **Complete pooling** — assume every manager has the same true rate. Ignores real differences between managers. - **No pooling** — trust every manager's raw rate exactly as calculated. Over-trusts the small teams shown above. - **Partial pooling** — each manager gets their own estimate, pulled toward the organisation-wide rate by an amount the data itself decides, based on `group_size`. ------------------------------------------------------------------------ ## Fitting the model This is count data with an exposure that varies hugely across managers — the right tool is a Poisson (or negative binomial) model with `group_size` entered as an **offset**, and a varying intercept per manager so the model can borrow strength across all 571 of them: Another log-scale intercept, so — same habit as Chapters 9 and 12 — translate it before trusting it: ```{r plot-prior-shrink} set.seed(13) tibble(log_intercept = rnorm(4000, -2.5, 1)) |> mutate(rate = exp(log_intercept)) |> ggplot(aes(rate)) + geom_histogram(bins = 40, fill = "#122a52", alpha = 0.6) + labs(title = "What Normal(-2.5, 1) on the log scale implies", subtitle = "Simulated draws from the intercept prior, translated to a per-person transfer rate", x = "Implied baseline transfer rate per team member", y = "Simulated draws") ``` ::: callout-note Centred around 8% (`exp(-2.5)`), with plenty of room either side — a sensible starting point for a rate we haven't looked at yet, and nothing like a confident claim about the true organisation-wide figure. ::: ```{r fit-shrink} fit_shrink <- brm( transfers ~ 1 + (1 | employee_id) + offset(log(group_size)), data = managers, family = poisson(), prior = c(prior(normal(-2.5, 1), class = Intercept), prior(exponential(2), class = sd)), chains = 4, iter = 2000, seed = 13, refresh = 0 ) summary(fit_shrink) ``` ::: callout-note ## Reading this formula `transfers ~ 1 + (1 | employee_id) + offset(log(group_size))` says: each manager's expected transfer count is their `group_size` (the offset — not something we're estimating, just accounting for) times a rate. That rate is the organisation-wide rate (`1`, the Intercept) plus a manager-specific adjustment drawn from a shared distribution (`(1 | employee_id)`) — exactly the varying-intercepts idea from Chapter 8, just applied one manager at a time instead of one property at a time. ::: ::: callout-note ## A different lens: this is a Bayesian version of a model you may already know If you've fitted a **mixed-effects / random-effects model** before — `lme4::lmer()` or `glmer()` in R, or their equivalents elsewhere — you've already used the frequentist cousin of this exact model. The `(1 | employee_id)` syntax is even identical. The classical version gives you a point estimate for each manager's random effect (a **BLUP**, "best linear unbiased predictor") and a standard error built on asymptotic approximations that get shaky for small groups — precisely the groups we care most about here. The Bayesian version gives every manager a full posterior distribution instead of a point-plus-error-bar, which turns out to matter exactly when a manager's team is small enough that "asymptotic" assumptions stop being a safe bet. Same shrinkage mechanism either way — the difference is what you get to say about the uncertainty afterward. ::: ::: callout-tip ## Only one row per manager — does that break anything? No. The model doesn't need *repeated* observations for each manager to learn from them collectively — it needs *many managers*, each contributing one data point of varying precision (set by `group_size`). It's the same logic as estimating 571 baseball players' true batting averages from a single season each: no one player gives you much, but the *pattern across all 571* tells the model how much any one manager's raw rate should be trusted. ::: ### What the group-level parameter tells you ```{r sd-check} sd_estimate <- summary(fit_shrink)$random$employee_id sd_estimate ``` `sd(Intercept)` here is a genuine finding: it says how much managers' *true* transfer rates actually vary, once you strip away the noise from different team sizes. A large value says management quality (or at least whatever drives transfer requests) varies a lot across the organisation; a small value says most of the raw league table's spread was noise all along. ------------------------------------------------------------------------ ### Diagnose, then check Two steps of the workflow before any estimate leaves this model, and they answer different questions. **Did the sampler work?** `summary()` above reports `Rhat` and `ESS` for every parameter. This model has a group-level `sd` estimated from many small groups, which is the geometry Chapter 11 warns about, so it is worth a deliberate look rather than a glance. **Can the model produce data like ours?** A count outcome needs the bar version of the check, as in Chapter 12: ```{r ppc-shrink} pp_check(fit_shrink, type = "bars", ndraws = 100) + labs(title = "Posterior predictive check", subtitle = "Bars = what the model predicts; points = what actually happened", x = "Transfer requests", y = "Number of managers") ``` The failure to watch for here is the one Chapter 12 named: if the observed counts in the right-hand tail sit above their bars, the Poisson is under-covering and the shrunk estimates will be too confident. The `negbinomial()` swap from Chapter 12 is the fix, and it changes the intervals rather than the ranking. ------------------------------------------------------------------------ ## Extracting each manager's shrunk rate To compare fairly, we ask the model for each manager's expected rate **per single person managed** — set `group_size` to 1 in new data so the offset drops out, while keeping each manager's own fitted adjustment: ```{r shrunk-rates} shrunk <- managers |> mutate(group_size = 1) |> add_epred_draws(fit_shrink, re_formula = NULL) |> mean_qi(.epred, .width = 0.95) |> rename(shrunk_rate = .epred, lower = .lower, upper = .upper) managers_compare <- managers |> select(employee_id, city, group_size, transfers, raw_rate, performance_group) |> left_join(shrunk |> select(employee_id, shrunk_rate, lower, upper), by = "employee_id") managers_compare |> arrange(desc(shrunk_rate)) |> head(10) ``` ### See the shrinkage ```{r shrinkage-plot} ggplot(managers_compare, aes(x = group_size)) + geom_hline(yintercept = sum(managers$transfers) / sum(managers$group_size), linetype = "dashed", colour = "#d32f2f") + geom_segment(aes(xend = group_size, y = raw_rate, yend = shrunk_rate), colour = "grey70", alpha = 0.6) + geom_point(aes(y = raw_rate), colour = "grey50", alpha = 0.5, size = 1.2) + geom_point(aes(y = shrunk_rate), colour = "#122a52", alpha = 0.6, size = 1.2) + scale_x_log10() + coord_cartesian(ylim = c(0, 0.4)) + labs( title = "Shrinkage across all 571 managers", subtitle = "Grey = raw rate, navy = model estimate, red dashed = org-wide rate", x = "Group size (log scale)", y = "Transfers per person managed" ) ``` Small-team managers move the furthest — often dramatically — while large-team managers barely move, because their own data is already trustworthy. ::: callout-important The manager with 3 reports and a 33% raw rate is not "the worst manager in the company" once you shrink properly — their small-team estimate gets pulled hard toward the organisational average, because three observations tell you almost nothing on their own. ::: ------------------------------------------------------------------------ ## Does the shrunk ranking agree with the official rating? `managers` also has a `performance_group` label (Bottom / Middle / Top performer) from an actual review process. A useful sanity check: where does the shrunk transfer-rate ranking disagree with that rating, and is disagreement more common for small-team managers? ```{r vs-performance-group} managers_compare |> group_by(performance_group) |> summarise( n = n(), mean_raw = mean(raw_rate), mean_shrunk = mean(shrunk_rate), mean_group_size = mean(group_size) ) ``` ```{r disagreement-plot} managers_compare |> mutate(gap = abs(raw_rate - shrunk_rate)) |> ggplot(aes(x = group_size, y = gap, colour = performance_group)) + geom_point(alpha = 0.6) + scale_x_log10() + labs( title = "Raw vs. shrunk gap, by team size and performance rating", subtitle = "The gap that shrinkage corrects is concentrated among small teams", x = "Group size (log scale)", y = "|raw rate − shrunk rate|", colour = "Performance rating" ) ``` ::: callout-caution ## Try it yourself Filter to managers where the raw rate would have put them in the "worst 10%" of the league table, but the shrunk rate does not. How many are there, and what does their `performance_group` say? This is precisely the group a naive league table would have unfairly flagged. ::: ------------------------------------------------------------------------ ::: callout-tip ## For the ML/DS crowd This exact problem — stabilising an estimate for an entity with too little of its own data, by borrowing strength from everyone else's — is also how production recommender systems handle new or low-activity users and items. A user with two ratings gets their predicted preferences shrunk toward the population average; one with two thousand ratings is trusted almost entirely on their own history. It's the same partial-pooling logic as this chapter, usually fitted with matrix factorisation and a regularisation term rather than `brms`, but solving an identical problem: don't let sparse data pretend to be confident. ::: ## Presenting this to a business audience that wants a league table anyway Stakeholders often want exactly the artefact this chapter argues against: a ranked list. You don't have to refuse — you have to change *what the list shows*. The deeper fix is to change what the list is *for*. A ranking invites the question "who is worst?"; a decision invites "who should we act on, and is acting worth it?" Chapter 24 takes these same posteriors and answers the second question, including the uncomfortable part — how many of the managers you pick will turn out to have been the wrong choice. ::: callout-note ## Three practical fixes for the deck 1. **Show the credible interval, not just a point estimate.** A manager whose 95% interval overlaps ten other managers' intervals is not meaningfully different from them — say so, rather than presenting rank 14 as distinct from rank 24. 2. **Report team size alongside the rank.** "Rank 3 of 571 (team of 6)" reads very differently from "Rank 3 of 571 (team of 84)" — let the audience see the input to their own scepticism. 3. **Group into tiers, not individual ranks.** "Clearly above organisational average," "not distinguishable from average," "clearly below average" — based on whether the credible interval excludes the organisation-wide rate — resists the false precision of rank 47 vs. rank 48. ::: ```{r tiering} managers_compare |> mutate( org_rate = sum(managers$transfers) / sum(managers$group_size), tier = case_when( lower > org_rate ~ "Clearly above average", upper < org_rate ~ "Clearly below average", TRUE ~ "Not distinguishable from average" ) ) |> count(tier) ``` This single table is usually the most useful thing you can hand a senior stakeholder: most managers, once you're honest about uncertainty, are **not distinguishable from average** — and that's a finding, not a disappointment. ------------------------------------------------------------------------ ## On the job {.unnumbered} ::: callout-important ## Why this matters day to day Any time you're asked to rank people-metrics across managers, teams, stores, or offices of different sizes — attrition, engagement, complaints, absence, sales per head — this chapter's approach applies directly. Publishing the raw league table is the single most common statistical mistake in People Analytics reporting, and it's an easy one to fix once you know to look for it: check whether the extremes cluster among the smallest groups. If they do, shrink before you share. ::: ------------------------------------------------------------------------ ## Summary {.unnumbered} ::: callout-note ## Today you learned 1. Raw per-manager rates are least trustworthy exactly where team size is smallest — and those are usually the ones that end up at the extremes of a league table. 2. A Poisson model with `group_size` as an **offset** and a varying intercept per manager lets you partially pool rates across an entire population of managers, even with one row per manager. 3. **Shrinkage** pulls small-team managers' rates toward the organisation-wide average, by an amount the model — not you — decides. 4. Comparing shrunk estimates to an existing performance rating is a useful sanity check, and often reveals that small-team managers were being mis-ranked. 5. You can still hand over a league table — just report **uncertainty and team size alongside rank**, or replace individual ranks with tiers. ::: ## Next chapter {.unnumbered} **Hierarchical models for pay and variance** — we take the same partial-pooling machinery and point it at a different, equally sensitive question: how much pay variation actually sits at the level, function, or location layer, versus being individual noise?