Our reader

How Rallybase reads public government pages, and how to tell us to slow down or stop.

Who we are

Rallybase keeps public records about who holds office in US government, from Congress to the county. Each record names the public page it was read from, and the day we read it where we hold that day.

Rallybase is independent and nonpartisan. It is not a government website.

How it names itself

Every request the reader that collects our records makes carries this user agent, with our contact address in it:

Mozilla/5.0 (compatible; Rallybase/1.0; civic data; contact support@rallybase.org)

How it reads

It asks for robots.txt first and follows it. If robots.txt answers with a server error or does not answer, it treats the site as closed and reads nothing there.

Each reader asks a site for one page at a time, and waits at least 3 seconds after one page before it asks that site for the next. Before the first page it asks for robots.txt, and it follows a redirect at once.

It does not retry a page that fails within a run. It does not log in, fill in forms or send anything. It does not solve or get around a bot check.

How often

Every day, scheduled jobs read the member pages of some state legislatures.

County and city pages are not read on a timer. We read them when we update that place, one read for each page.

When a site says no

If your site refuses our reader (HTTP 401, 403 or 429), we stop asking it, and ask again no sooner than 7 days later.

We do not read past a bot check or a robots.txt rule against our agent. A notice asking automated readers to keep out, or a message from you, stops us until you write to say otherwise.

Our address

We do not read from a fixed IP address yet. If your site would allow our reader by its address, write to us and we will reply when a fixed, named address exists.

Ask us to slow down or stop

To ask us to slow down, to read a different file or feed, or to stop reading your site, write to support@rallybase.org. Please write from your office's own address, so we know the request is yours. We stop reading the site and reply to confirm.