UNIVAF is a system for gathering vaccination appointment availability information from providers across North America and making it available in a standard format via a free-to-use, open API. It acts as the source of data for both government- and community-run vaccine finders, such as the State of New Jersey’s vaccine finder and Vaccinate the States. You can access the live API and documentation at https://getmyvax.org/. The live service has been shut down, but you can view historical data and documentation at https://archives.getmyvax.org/.
While currently focused on COVID-19 vaccinations, we hope the code and infrastructure here might be easily repurposed in the future for other kinds of everyday vaccinations (e.g. flu vaccines) or for future health emergencies.
UNIVAF is built in Node.js and TypeScript.
Table of Contents
This project is broken up into two major components, each in a separate directory:
-
serveris a small API server that wraps a Postgres database. Various scrapers and API clients canPOSTappointment data to it, and consumers canGETdata from it. It’s currently accessible in production at https://getmyvax.org/. -
loaderis a set of scrapers and API clients that discover information about vaccine provider locations and appointment availability, then send it to the server. We currently run them on a schedule every few minutes (seeterraform/loaders.tf), but they can also be run in a loop, as a server, or whatever works best.
At the top level of this repo, you’ll also find some other useful directories:
-
commonis shared library code that is used by both the server and loader components. -
archivescontains documentation for the the archived historical data the server saves as an open dataset, and which can be used for deeper analysis of vaccination reach, equity, etc. It's ultimately served from https://archives.getmyvax.org/. -
docscontains internal project documentation for contributors, infrastructure guidance, and incident reports. -
scriptscontains scripts for managing releases, deployments, and other tasks. -
terraformcontains Terraform configuration files used to deploy this project to AWS. We try to keep as much of our infrastructure configuration as possible stored here as code. -
tombstonecontains a placeholder page that is deployed in place of the production API (since it's been shut down). If you are forking and re-deploying this project, you'll probably want to remove it.
You will need to download the latest version of Docker to run Postgres.
$ make docker # makes the cross-OS compatible docker image
$ make compose # runs the database as a background process on 5432
$ pgcli -p 5432 -h 127.0.0.1 postgres postgres # connect locally and verify (optional)
$ postgres@127:postgres> \l # list the databases (optional)
+-----------+----------+------------+------------+------------+-----------------------+
| Name | Owner | Encoding | Collate | Ctype | Access privileges |
|-----------+----------+------------+------------+------------+-----------------------|
| postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | <null> |
| template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres |
| | | | | | postgres=CTc/postgres |
| template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres |
| | | | | | postgres=CTc/postgres |
+-----------+----------+------------+------------+------------+-----------------------+Next, seed the database and run the server (see below)!
$ npm install
$ cd ./server/
$ npm run db:seed # `db:seed` will populate the database with example data for local development
$ npm run watch # `watch` will auto-recompile typescript
$ npm run test # `test` will run the various jest tests
$ open http://localhost:3000/providersTo install, run npm install in the loader directory:
$ npm install
$ cd ./loaderThen load data from any supported sources by running bin/univaf-loader with a list of the sources you want to load data from:
# Load data from NJVSS and the CVS SMART API
$ bin/univaf-loader njvss cvsSmartUse --help to see a list of sources and other options:
$ bin/univaf-loader --help
univaf-loader [sources..]
Load data about COVID-19 vaccine appointment availability from a
variety of different sources.
Data about each vaccination location is written as a single line of JSON
on STDOUT, so you can pipe or stream the output to other files or
programs for processing. Informational messages are available on STDERR.
Supported sources: albertsonsMhealth, albertsonsScraper, cdcApi, cvsApi,
cvsScraper, cvsSmart, heb, hyvee, krogerSmart, njvss, prepmod, riteAidApi,
riteAidSmart, riteAidScraper, vaccinespotter, vtsGeo, waDoh, walgreensSmart
Exit codes:
- 90: An unhandled error occurred.
- 91: An error occurred in one of the requested sources.
- 92: An error occurred in all of the requested sources.
Options:
--version Show version number [boolean]
--help Show help [boolean]
--send Send availability info to the API specified by the
environment variable API_URL. If set, data will not
be written to STDOUT. [boolean]
--compact Output JSON as a single line [boolean]
--states Comma-separated list of states to query for
multi-state sources (e.g. cvsSmart). If not
specified all relevant states for the requested
sources will be checked. [string]
--hide-missing-locations If a previously found location stops being returned
by a source, output it with `is_public: false`. Only
use this with sources that are "authoritative" --
that is, you expect them to output a *complete* list
of whatever type of locations they cover.
("Previously found" locations are loaded from the
server specified by the API_URL environment
variable. This is currently only supported by the
`prepmod` source.) [boolean]
--rate-limit Only make this many HTTP requests per second. (Only
applies to the Rite Aid sources for now.) [number]
--filter-stale-data Don't report records with stale data. [boolean]
--stale-threshold Consider records older than this many milliseconds
to be stale. [number] [default: 86400000]Please see the deployment section of the runbook.
This repo falls under U.S. Digital Response’s Code of Conduct, and we will hold all participants in issues, pull requests, discussions, and other spaces related to this project to that Code of Conduct. Please see CODE_OF_CONDUCT.md for the full code.
This project wouldn’t exist without the hard work of many people. Thanks to the following for all their contributions! Please see CONTRIBUTING.md to find out how you can help.
| Contributions | Name |
|---|---|
| 💻 🚇 | Jacob Aronoff |
| 💻 |
Rob Brackett |
| 🤔 💻 |
Dave Cole |
| 💻 |
Nelson Elhage |
| 💼 | Mike Flowers |
| 💻 |
Calvin French-Owen |
| 🔬 | Tom MacWright |
| 💻 | Chantel Miller |
| 🤔 | Giuseppe Morgana |
| 💻 |
Aston Motes |
| 📆 | Emilia Ndely |
| 💻 |
Alan Ning |
| 🔬 💻 | Jan Overgoor |
| 💻 |
Christina Roberts |
| 🔬 📓 | Mollie Ruskin |
| 💻 | Greg Sandstrom |
| 💻 |
Stephan Schmidt |
| 💻 |
Sam Szuflita |
| 💻 | Jesse Vincent |
| 🤔 📆 | Diana Wang |
(For a key to the contribution emoji or more info on this format, check out “All Contributors.”)
Copyright (C) 2021–2023 U.S. Digital Response (USDR)
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License. You may obtain a copy of the License at:
LICENSE in this repository or http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.