Open Telugu dataWhat the apps compute with, and where it came from

Open Telugu data format, version 1

The one format of every file data.opentelugu.org publishes under /v1/, whatever its kind and whichever app reads it. A reader that follows this document can read every kind’s envelope, verify it, and show its source; a kind’s own document adds only its key rule and its data fields. It is shown on the site at /en/format/ (with a Telugu summary at /te/format/).

The words MUST, MUST NOT and MAY are used as in RFC 2119.

1. Addresses

Address What
/v1/index.json the index: every kind, its format and check rule, every key’s latest revision (§5)
/v1/files.json every revision ever published, with sha256 and size (§5)
/v1/<kind>/<key>.r<revision>.json one revision of one key; never changes once published

kind and key match [a-z0-9]+(-[a-z0-9]+)*; revision is an integer from 1. Nothing else is published under /v1/ except what a later version of this document adds.

2. Versions

3. The envelope

Every revision file is one JSON object with these fields, in this order, and the kind’s fields in data:

Field Type Rule
kind string equals the address’s kind
format integer the kind’s format
key string equals the address’s key; matches the kind’s key pattern
revision integer equals the address’s revision
title text (§4) what this file is, for people
covers object or null { "from": date, "to": date }, inclusive: the period the key is for (a season, a year); null when the key is not a period
status string checked: compared with the official source itself; provisional: from secondary sources, not yet compared
sources array each { "id", "body", "title", "url", "date" } (date of publication); the official source first when there is one
taken date when the figures were taken from the sources
checked object or null { "date", "how" }; required when status is checked, else null
to_check string or null the official source still to compare with; required when status is provisional, else null
licence object { "data": "CC BY 4.0", "holder": "Open Telugu", "note": … }: the licence of this compilation and the terms of the sources
changes object or null null in revision 1; from revision 2: { "date", "why", "items": [ { "path", "from", "to" } ] } (§6)
data object the kind’s own fields, defined by its document

A reader MUST refuse a file whose kind, format, key or revision differ from what it asked for, whose status is not one of the two, or whose required fields are missing. Readers MAY use provisional data only when they show it as not yet confirmed, beside its source.

4. Conventions

5. Index and files list

{
  "format": 1,
  "built": "2026-10-30T10:12:00Z",
  "core": "<full commit of @opentelugu/core>",
  "kinds": {
    "<kind>": {
      "format": 1,
      "title": { "te": "…", "en": "…" },
      "check": { "days": 7, "daily": [["05-01", "07-15"]] },
      "keys": {
        "<key>": { "revision": 1, "path": "/v1/<kind>/<key>.r1.json", "sha256": "<hex>", "size": 4210, "date": "2026-10-28", "status": "provisional" }
      }
    }
  }
}

6. Corrections

A correction is a new revision; the old one stays published unchanged. Anything in the file may be corrected: a figure in data, or the envelope (a source’s link, a title’s translation, the licence note, the status). changes.items lists exactly the values that differ from the revision before, apart from revision and changes themselves, each with a path: segments joined by / from the file’s top, where an array of objects is entered by id (data/crops/paddy_common/msp_per_quintal, sources/agrospectrum/url, title/te, status), and an added or removed value has from or to null. A revision MUST change something, so items is never empty. Core’s tests compute the differences and fail if items disagree.

A reader comparing a saved figure compares it with the latest revision it holds at the same path, not with the changes of the revisions in between, so a reader that skipped revisions is still right.

7. Reading, for any app

  1. Ship the published files an app needs, byte for byte, with the index they came from.
  2. Resolve the key with core’s resolver for the kind, with the inputs and within the geography its kind document states; never “the latest”.
  3. Use the highest verified revision held of exactly that key; if none, say it is not available yet.
  4. When a check is due (§5): fetch the index (conditionally); for every key held, kept or requested and not yet received, fetch revisions higher than held; accept a file only when status 200, size, sha256 and the envelope’s identity all match. A check succeeds only when the index and every file it called for are verified and durably stored.
  5. Keep with anything saved the key, revision and path its figures came from; compare with the latest revision held (§6).

Core’s data-client.js does all of this for any kind.

8. Privacy

Readers send only GET requests for published addresses. Apps outside a browser MAY send If-None-Match; browsers leave revalidation to their own cache (cache: 'no-cache'), so no request needs a CORS preflight. No query strings, cookies or identifiers are used; the site keeps nothing about readers. Responses carry Access-Control-Allow-Origin: * and Access-Control-Expose-Headers: ETag.

9. A kind’s document

Each kind has docs/kinds/<kind>.md stating: what it is for; its key, key pattern and resolver, with the resolver’s inputs and the geography it is supported for; what covers means; its data fields with types and units; its check rule; its sources; its format history.