Recipes¶
Useful Elex patterns. Contribute your own.
Filter with jq and upload to S3¶
This recipe uses the jq json filtering tool to create a national results json data file with a limited set of data fields and the AWS cli tools to upload the filtered json to S3.
Requirements:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | #!/bin/bash
# S3 url: MUST be set to your bucket and path.
ELEX_S3_URL='mybucket.tld/output/path.json'
# Get results and upload to S3
elex results 2012-11-06 -o json \
| jq -c '[
.[] |
select(.level == "state" ) |
select(.officename == "President") |
{
officename: .officename,
statepostal: .statepostal,
first: .first,
last: .last,
party: .party,
votecount: .votecount,
votepct: .votepct,
winner: .winner,
level: .level
}
]' \
| gzip -vc \
| aws s3 cp - s3://$ELEX_S3_URL \
--acl public-read \
--content-type=application/json \
--content-encoding gzip
# Check response headers
curl -I $ELEX_S3_URL
# Get first entry of uploaded json
curl -s --compressed $ELEX_S3_URL | jq '[.[]][0]'
|
ELEX_S3_URL
must be set to your s3 bucket and path.
Steps:
- Get election results in json format with
elex
- Pipe results to
jq
for filtering - Pipe filtered results to
gzip
to compress - Pipe gzipped results to
aws s3 cp
to send to S3.
Inspect with an ORM using Flask and Peewee¶
This recipe uses the Flask web framework and the Peewee Python ORM to model, query and update data that elex
provides.
Requirements:
- Elex loader, an NYT project that calls
elex
to load data into a Postgres database with CSV and the PostgresCOPY
command. - Elex admin, an NYT project that is a simple, web-based admin for creating and editing data to override AP election results, including candidate names, race descriptions, and race calls.
Steps:
- Install
elex-loader
using these instructions. - Install
elex-admin
using these instructions.
Extra steps:
- Use the
models.py
that come withelex-admin
to query data.