Add jurisdiction info to the individual data records?

[jessemortenson]

Would it be easy to add jurisdiction data to the individual data records in the data folder? I can write a script to grab that data from the file structure and inject it into a modified version of the records, but it's extra work :) One lesson I've had from some of the legacy Open States stuff is that splitting the true data about a record between the individual file's data (x.json) and the file structure in which the file sits (ca/anaheim) causes some headaches.

In my mind we'd have additions of state and place properties (or equivalent labels):

- name: Ashleigh E. Aitken
  state: CA
  place: Anaheim
  roles:
  - Mayor
  divisions: []
  image: https://www.anaheim.net/ImageRepository/Document?documentID=58010
  cdn_image: https://cdn.civicpatch.org/open-data/ca/anaheim/images/4c64f17f0de92d0b4499d1aca906a03741fd22e571b0e40fd4b45b53d0a4d258.jpg
  email: aaitken@anaheim.net
  phone_number: "(714) 765-5247"
  website: https://www.anaheim.net/5174/Anaheim-Mayor-Ashleigh-Aitken
  start_date: 2022-11
  end_date:
  sources:
  - https://www.anaheim.net/5174/Anaheim-Mayor-Ashleigh-Aitken
  - https://www.anaheim.net/173/City-Council
  - https://www.anaheim.net/5175/Council-Member-Natalie-Meeks
  - https://www.anaheim.net/3522/Council-Member-Jose-Diaz
  - https://www.anaheim.net/2314/Council-Member-Carlos-A-Leon
  - https://www.anaheim.net/3523/Council-Member-Natalie-Rubalcava
  - https://www.anaheim.net/3524/Mayor-Pro-Tem-Norma-Campos-Kurtz
  - https://www.anaheim.net/3521/Council-Member-Stephen-Faessel
  updated_at: '2025-07-17T17:22:50Z'

[shelltr]

Sure, yes. I'll add it. I owe you a model for this as a part of/after the rewrite.

https://github.com/CivicPatch/civicpatch-tools/tree/python-rewrite

[shelltr]

@jessemortenson Given that the jurisdiction ids are hierarchical for places vs cousubs (ex: I anticipate ocdids to look like this:
ocd-jurisdiction/country:us/state:wa/place:seattle
or
ocd-jurisdiction/country:us/state:il/county:dupage/place:naperville)

What do you think of the following change to add jurisdiction_id as an entry in place of the state + place fields?

Additionally, to avoid name collisions, I'm planning on changing the folder structure from mostly flat to match the jurisdiction ids more closely (basically follow whatever nesting jurisdiction ids have)

So people.yml output for:
ocd-jurisdiction/country:us/state:il/county:dupage/place:naperville
Turns into
data/il/dupage/naperville/people.yml

This way I don't really have to think about name collisions since jurisdiction ids are guaranteed to be unique.

[jessemortenson]

Thanks, Michelle! Yes I think the jurisdiction ID works great, good idea!

[shelltr]

Sorry! Still thinking about the folders output. Just realized the scheme of following the jurisdiction hierarchy is kind of weird because it puts places & counties at the same level when you're just browsing the data.

What do you think of this:

ocd-jurisdiction/country:us/state:il/county:dupage/place:naperville
data/il/counties/dupage/places/naperville/people.yml

&

ocd-jurisdiction/country:us/state:il/place:naperville
data/il/places/naperville/people.yml

so when you mix them together you'll have

more nesting

data/
--> il/
----> counties/
------> naperville_county/
--------> little_naperville_city/
----------> people.yml
----> places/
------> naperville_city/
--------> people.yml

instead of

confusing

data/
--> il/
----> naperville_city
------> people.yml
----> naperville_county
------> little_naperville_city
--------> people.yml

Let me know what you think

[jessemortenson]

Honestly, at the end of the day this decision isn't crucial for my purposes, because as long as the data exists on a file somewhere in this structure, and each record inside the data contains metadata (I can get jurisdiction ID from the data, rather than having to parse folder structure for that data), then I'm gonna be ok. And I think most folks using this as structured data will be the same. (eg we're just recursively grabbing all yml files in the folder structure, and putting all that data into a single data frame or SQL db or whatever).

That said I don't mind providing an opinion! I think my vote would rather be for a relatively flat directory structure, with more files in each folder, rather than a structure that attempts to precisely nest. I think for someone evaluating the data at first glance to be able to scan visually and be able to kinda see what's there. So I would not vote to hierarchically nest cities/places under their counties. It makes it harder IMO to scan and see "what is the extent of this data" without having to click into lots of folders to see what is in each.

For ex, why have a file called "people.yml" in a folder called "little_naperville_city"? Usually a directory that contains only one file is not super useful (directory implies there are multiple files inside, IMO, like a directory of people or offices lists multiple people or offices). Are you planning to eventually have other data types, ie people.yml, elections.yml, retired_people.yml etc.? I might just not be aware of the vision.

Honestly I'd probably just use a naming convention so everything is pretty flat within a state/province/territory, like:
data/

--> il/
----> county_naperville.yml
----> place_little_naperville_city.yml
----> place_naperville_city.yml

[shelltr]

Hmmmmm good point about the folder navigation. No, there will only ever be people data, maybe under different types (to accommodate candidate filings)

I'll get rid of the nested data but rename the files so that they follow whatever jurisdiction id you plug into it.

--> il/
----> ocd-jurisdiction/country:us/state:il/county:naperville/council --> county_naperville.yml
----> ocd-jurisdiction/country:us/state:il/county:naperville/place:little_naperville_city/council --> county_naperville__place_little_naperville_city.yml
----> ocd-jurisdiction/country:us/state:il/place:naperville_city/council --> place_naperville_city.yml

So no diff for when you parse it, but also very clear on place vs cousubs based on the file names.

And candidate filings/future people data types can live under the same files as the elected representatives at the same level, with maybe different ocdids (maybe ocd-jurisdiction/country:us/state:il/county:naperville/candidate_filings instead of council)