What We've Learned (So Far)

What we've learned about California's covid-19 response effort in our first 42 days.

VaccinateCA organized 42 days ago to help Californians get accurate information about where the covid-19 vaccines were available. We collect it by calling hundreds of medical professionals daily and writing down what they tell us, then publishing it to our site and otherwise distributing it. We are a non-profit, non-partisan effort, interested only in improving Californians’ ability to access the vaccines which society’s larger response effort has made available.

We are far from experts about this problem, but we have learned a bit in the last 42 days, and wanted to record it here. Hopefully it will help others grappling with these issues to accelerate their own responses. Every day matters, and every dose matters, in the fight against the pandemic.

No one has a comprehensive view of vaccine distribution.

The nature of the vaccine rollout in the United States makes literally thousands of organizations responsible for producing, tracking, delivering, stockkeeping, administering, and recordkeeping the vaccine supply. Overwhelmingly, these organizations do not have defined interfaces for data flow between themselves. They often have internal procedures for tracking stock availability which do not match with the questions people want to ask about covid-19 vaccine availability specifically, and as such cannot provide accurate organizational-level answers for questions society has right now.

Because many organizations responsible for distributing the vaccine cannot report their current status within the organization, organizations conceptually “above” them in the supply chain cannot produce reliable and accurate aggregates. This regrettably includes some numbers which various stakeholders treat as reliable and accurate aggregates.

These are not indictments of anyone. We face the pandemic with the systems and processes which we have, not the ones we wish we had.

The ground truth is out there: The most accurate understanding of how many doses are available at a location is invariably available to the medical professional who will physically inject the vaccine. They also have an accurate understanding of who they are allowed, by the policy which controls locally, to inject. We favor information collected from frontline workers versus intermediaries because it aligns with our ultimate objective, getting Californians accurate information about where they could be vaccinated today.

If a shipping manifest tells you something and the pharmacist tells you the opposite about a vaccine’s availability at a pharmacy, the pharmacist is almost always right, and as a vaccine seeker or care manager you should always favor the pharmacist’s version of the truth. We help Californians do that, at scale, while lessening the load on the healthcare system answering the same questions redundantly.

The situation on the ground is heterogenous, changes quickly, and does not trend simply.

It is widely known in the healthcare and advocacy communities that access to healthcare is not uniform across the population, and indeed that would be an undeniable conclusion of the data we see daily. There are substantial geographical, socioeconomic, and other disparities in the vaccine distribution across the state of California, in many ways mirroring pre-existing disparities in the distribution of healthcare providers across the state of California. Policymakers and the healthcare community are incredibly aware of this fact, and have spent much of their efforts this past year on preparing for it.

The situation on the ground, though, is even more complicated than one would suppose. There can exist pockets of unexpected availability due to e.g. a local decision made by a single EMT at a fire department, or pockets of unexpected failure-to-satisfy-appointments due to e.g. a single waylaid vaccine shipment, which might ripple through hundreds of appointments at dozens of organizations.

We detected substantial systemwide disturbances in vaccine availability during the inclement weather of the last few weeks, as expected shipments were delayed or mislaid, causing expected appointments to be canceled or rescheduled. Because the vaccination effort is a complicated system, these tend to affect not just the directly-implicated providers but also ones in their communities, as e.g. vaccine seekers spill over to places where there is still supply, etc.

The supply chain for the vaccine has less redundancy in it than most supply chains because it intentionally tries to run with very little inventory. This is the correct choice; society should prefer shots-in-arms over shots-on-shelf. It does mean, however, that hiccups which many supply chains would deal with without incident will bubble up to either end-users or system decision makers, very quickly. The system as currently constituted has little way of tracking that.

The complexity of this problem, and the way it defies simple narratives like “Supply is increasing”, makes it urgent that we get a handle on ground truth and keep a close eye on it, even as (and especially as) the situation “looks like it is getting better.”

The resourcing requirements to implement this model are high.

There have been many volunteer-led projects aimed at improving society’s response to the pandemic. We welcome all of them, in the same way that we cheer for every success of more formal response efforts. There are only two sides here, and the other side is the virus.

That said, we want to provide some commentary so interested parties understand the scale of effort building and maintaining a service like VaccinateCA requires. We have been described as a “crowdsourcing” effort. This is a misnomer; we are a professionalized operation, operating at roughly the scale of a Series A / Series B startup. We make hundreds of calls daily via volunteers, but we also have paid callers, paid technical and operations staff, and a call center making calls on our behalf.

We do not want to discourage anyone from attempting to intervene against the pandemic, but wanted to signpost for future civic entrepreneurs. The cost and complexity of web technology has fallen precipitously over the last few years, and publishing information to the Internet is easily within the reach of motivated individuals. Collecting that information can be very costly, particularly if one cannot simply reuse work of well-resourced organizations such as open public data sources.

Society must collaborate in the response effort

We are privileged to have the opportunity to work with medical providers, frontline workers, government decisionmakers, the scientific community, and other private efforts in society's collective response to the covid-19 pandemic. Each of us has a part in ending this crisis. We are moving as quickly as we can against our small part of this puzzle, and urge everyone else to do the same.

Interested in supporting our efforts? Sign up here to volunteer: https://vaccinateca.com/volunteer