Batch geocoding a CSV file

This page geocodes a whole CSV file of addresses with one Python script. It adds the latitude, longitude, matched address, and match type to every row, and writes the result to a new file. It uses only Python's standard library, so there's nothing to install beyond Python 3.

Mapricot has no upload or batch endpoint. The script sends one search request for each row, a few at a time. Each row is one request: the first 250,000 every 30 days are free, then they cost $0.10 per 1,000. See pricing.

1. Get a key

Create a server key with the places scope in the dashboard, and set it as an environment variable:

export MAPRICOT_KEY="mapricot_sk_…"

2. Save the script

Save this as geocode_csv.py:

import csv
import json
import os
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from concurrent.futures import ThreadPoolExecutor

KEY = os.environ["MAPRICOT_KEY"]
EMPTY = {"lat": "", "lon": "", "label": "", "type": ""}


def search(address):
    url = "https://api.mapricot.com/v1/search?" + urllib.parse.urlencode({"q": address, "limit": 1})
    request = urllib.request.Request(url, headers={"Authorization": f"Bearer {KEY}"})
    for attempt in range(4):
        try:
            with urllib.request.urlopen(request, timeout=30) as response:
                return json.load(response)["results"]
        except urllib.error.HTTPError as error:
            if error.code not in (429, 503) or attempt == 3:
                raise
            time.sleep(2**attempt)


def geocode(row):
    address = row["address"].strip()
    if not address:
        return EMPTY
    results = search(address)
    if not results:
        return EMPTY
    match = results[0]
    return {"lat": match["lat"], "lon": match["lon"], "label": match["label"], "type": match["type"]}


source, target = sys.argv[1], sys.argv[2]
with open(source, newline="", encoding="utf-8-sig") as file:
    reader = csv.DictReader(file)
    fields = [*reader.fieldnames, *EMPTY]
    rows = list(reader)

with open(target, "w", newline="", encoding="utf-8") as file, ThreadPoolExecutor(max_workers=4) as pool:
    writer = csv.DictWriter(file, fields)
    writer.writeheader()
    for done, (row, match) in enumerate(zip(rows, pool.map(geocode, rows)), 1):
        writer.writerow({**row, **match})
        if done % 100 == 0 or done == len(rows):
            print(f"{done} of {len(rows)} rows", file=sys.stderr)

3. Run it

The input needs a header row with an address column. Other columns are copied through as they are:

id,address
1,"128 SW 3rd Ave, Portland, OR"
2,"10 Downing Street, London"

python3 geocode_csv.py addresses.csv geocoded.csv

The output has the same rows in the same order, with four columns added:

Rows with an empty address, or no match, get empty columns.

Check the matches

Geocoding works best on complete addresses with a city and country. Before using the results, look at the rows whose type isn't address. A locality match means only the town was found, so the point is the middle of town, not the building. Comparing label with the original address shows what was matched.

Adjust it

• Addresses split over several columns. Join them in geocode, for example address = ", ".join([row["street"], row["city"], row["postcode"]]).
• A different column name. Change row["address"].
• Addresses in known countries. Add "country": "US", or a list such as "US,CA", next to "limit": 1 in search. It stops a "Portland" from matching the wrong country.
• Speed. The script sends 4 requests at a time. Each account has a limit on requests in progress at once, and requests over it get 429, which the script waits on and retries. Errors aren't billed, so retries cost nothing.

If a request fails for any other reason, the script stops with the HTTP error, such as 401 for an invalid key. The rows before it are already in the output file.

Questions

Can I upload a file to Mapricot instead? Not yet. Every request goes through the API, which this script calls for you.

How long does it take? It depends on your connection and the addresses. The script prints its progress every 100 rows.

Can I keep the results? Yes. Mapricot's results are built on open data, and you can store them permanently. Show the data credits if you display them: see geocoding.

Can I do this in a spreadsheet? Yes, for smaller lists. See geocoding in Google Sheets and Excel. For a script in your own code, see Python or Node.js.