Skip to main content
PermitCore refreshes permit data nightly from each metro’s open-data portal. Cohort classification, DQ flagging, and parcel join happen in the same nightly pipeline.

​
Refresh cadence

  • Source pull: every metro’s open-data portal is pulled once per 24 hours, scheduled per metro to land before 02:00 local time.
  • Cohort classify + DQ flag: runs immediately after each source pull. Adds ~15 minutes per metro for the classifier pass.
  • Site + API publish: updated rows land in the API by 02:30 local time per metro. The cohort distribution endpoint caches 1 hour (Cache-Control: max-age=3600).

​
The as_of_utc field

Every API response that returns aggregated state includes an as_of_utc timestamp marking when the underlying data was last computed:
Use this to detect stale cached responses, drive your own UI freshness indicators, and reconcile multi-source pipelines.

​
ISR caching on the site

The PermitCore site (permitcore.io) consumes the same cohort distribution endpoint via Next.js ISR with revalidate: 3600. So the site’s cohort breakdowns lag the underlying data by at most 1 hour beyond the upstream pipeline’s own latency. For real-time-critical use cases (e.g., webhook-driven CRM lead loading), use the /v1/permits endpoint directly. A webhook event stream is on the roadmap and not yet available.

​
Field-level data-quality flags

Every permit row carries a 12-key dq_flags object surfaced per field. The flags come from the WS-PE-DQ-2 bounds-rule registry and tell you which fields the pipeline’s rules flagged as suspect — false on a flag means the rule has not flagged this value, NOT the value was independently verified clean (the SD-039 semantic). NULL silver flag values are coerced to false per the same SD-039 rule. All 12 keys are always present when dq_flags is non-null. The dq_flags object is emitted today on both GET /v1/permits and GET /v1/permits/by-canonical-uid/{canonical_uid}, landed with the 2026-06-04 enrichment promotion — see the changelog. For NYC permits the detail endpoint also returns a nested pluto object ({is_placeholder, oob}) reporting the PLUTO parcel-data presence + DQ status with the same SD-039 semantic. That object is list-endpoint NULL by design — the PLUTO JOIN adds ~75ms p95 and disqualifies for the browse path.

​
How freshness interacts with your billing

Tier-aware quotas reset on the first of each month UTC. Your usage counters update in near-real-time on cache invalidation (typically within 10 seconds of a successful request). See Rate limits → for current tier quotas.