1. Overview

The WIS2 Downloader exposes a REST API for managing subscriptions and monitoring the system.

2. Authentication

Currently, the API does not require authentication. In production, consider placing it behind a reverse proxy with authentication.

3. Subscriptions API

3.1. List Subscriptions

Returns all active subscriptions.

GET /subscriptions

3.1.1. Response

Status Description

200

Success - returns map of topic to subscription details

503

Redis unavailable

3.1.2. Example

curl http://localhost:5002/subscriptions
{
  "cache/a/wis2/+/data/core/weather/surface-based-observations/#": {
    "topic": "cache/a/wis2/+/data/core/weather/surface-based-observations/#",
    "save_path": "surface-obs",
    "filters": {
      "accepted_media_types": ["application/bufr", "application/grib"]
    }
  }
}

3.2. Create Subscription

Creates a new subscription to a WIS2 topic.

POST /subscriptions
Content-Type: application/json

3.2.1. Request Body

Field Type Required Description

topic

string

Yes

WIS2 MQTT topic pattern. Supports + (single-level) and # (multi-level) wildcards.

target

string

No

Subdirectory under /data where files will be saved.

filters

object

No

Filter configuration.

filters.accepted_media_types

array

No

List of allowed MIME types. Supports wildcards (e.g., application/x-grib*).

3.2.2. Response

Status Description

202

Accepted - subscription queued for processing

400

Bad request - missing or invalid topic

503

Service unavailable - Redis connection failed

3.2.3. Response Headers

Header Description

Location

URL to retrieve subscription details

3.2.4. Example

curl -X POST http://localhost:5002/subscriptions \
  -H "Content-Type: application/json" \
  -d '{
    "topic": "cache/a/wis2/+/data/core/weather/surface-based-observations/#",
    "target": "surface-obs",
    "filters": {
      "accepted_media_types": ["application/bufr", "application/grib"]
    }
  }'
{
  "status": "accepted",
  "topic": "cache/a/wis2/+/data/core/weather/surface-based-observations/#",
  "target": "surface-obs"
}

3.3. Get Subscription

Returns details for a specific subscription.

GET /subscriptions/{topic}

3.3.1. Path Parameters

Parameter Type Description

topic

string

WIS2 topic (only # requires encoding as %23)

3.3.2. URL Encoding

Only the # character requires URL encoding:

Character Encoded Reason

#

%23

Otherwise interpreted as URL fragment
Slashes (/) and + do not require encoding in the path.

3.3.3. Response

Status Description

200

Success

400

Invalid topic

404

Subscription not found

503

Redis unavailable

3.3.4. Example

curl "http://localhost:5002/subscriptions/cache/a/wis2/+/data/core/weather/surface-based-observations/%23"
{
  "topic": "cache/a/wis2/+/data/core/weather/surface-based-observations/#",
  "save_path": "surface-obs",
  "filters": {
    "accepted_media_types": ["application/bufr", "application/grib"]
  }
}

3.4. Delete Subscription

Removes a subscription.

DELETE /subscriptions/{topic}

3.4.1. Path Parameters

Parameter Type Description

topic

string

URL-encoded WIS2 topic

3.4.2. Response

Status Description

200

Successfully deleted

400

Invalid topic

503

Redis unavailable

3.4.3. Example

curl -X DELETE "http://localhost:5002/subscriptions/cache/a/wis2/+/data/core/weather/surface-based-observations/%23"
{
  "status": "deleted",
  "topic": "cache/a/wis2/+/data/core/weather/surface-based-observations/#"
}

4. Monitoring API

4.1. Health Check

Returns service health status.

GET /health

4.1.1. Response

Always returns 200. Check the status field for actual health.

Field Values

status

healthy or unhealthy

4.1.2. Example

curl http://localhost:5002/health
{"status": "healthy"}

4.2. Prometheus Metrics

Returns metrics in Prometheus text format.

GET /metrics

4.2.1. Response

Status Description

200

Prometheus text format metrics

500

Error generating metrics

4.2.2. Available Metrics

Metric Type Labels Description

wis2downloader_notifications_total

Counter

status

Total notifications processed

wis2downloader_downloads_total

Counter

cache, media_type

Successfully downloaded files

wis2downloader_downloads_bytes_total

Counter

cache, media_type

Total bytes downloaded

wis2downloader_skipped_total

Counter

reason

Skipped notifications by reason

wis2downloader_failed_total

Counter

cache, reason

Failed downloads

wis2downloader_celery_queue_length

Gauge

queue_name

Current tasks in Celery queue

4.2.3. Example

curl http://localhost:5002/metrics
# HELP wis2downloader_downloads_total Total number of files downloaded.
# TYPE wis2downloader_downloads_total counter
wis2downloader_downloads_total{cache="example.cache.com",media_type="application/bufr"} 42.0

# HELP wis2downloader_notifications_total Total notifications processed.
# TYPE wis2downloader_notifications_total counter
wis2downloader_notifications_total{status="success"} 42.0
wis2downloader_notifications_total{status="skipped"} 5.0

4.3. OpenAPI Specification

Returns the OpenAPI 3.0 specification as JSON.

GET /openapi

4.4. Swagger UI

Interactive API documentation.

GET /
GET /swagger

5. Error Responses

All error responses follow this format:

{
  "error": "Error message describing the problem"
}

5.1. Common Error Messages

Status Error Cause

400

"No topic provided"

POST /subscriptions without topic field

400

"No topic passed"

GET/DELETE with empty topic

404

"Subscription for topic X not found"

Topic not subscribed

503

"Failed to connect to Redis: …​"

Redis unavailable

503

"Failed to queue subscription command…​"

Redis pub/sub failed

503

"Failed to persist subscription…​"

Redis write failed

6. Data Schemas

6.1. Subscription

{
  "topic": "string (required) - WIS2 MQTT topic pattern",
  "save_path": "string | null - Subdirectory for downloads",
  "filters": {
    "accepted_media_types": ["array of MIME type strings"]
  }
}

6.2. Filters

{
  "accepted_media_types": [
    "application/bufr",
    "application/grib",
    "application/x-netcdf*"
  ]
}

6.2.1. Default Accepted Media Types

When no filter is specified, these types are accepted:

[
  "image/gif", "image/jpeg", "image/png", "image/tiff",
  "application/pdf", "application/postscript",
  "application/bufr", "application/grib",
  "application/x-hdf", "application/x-hdf5",
  "application/x-netcdf", "application/x-netcdf4",
  "text/plain", "text/html", "text/xml", "text/csv",
  "text/tab-separated-values"
]

7. Topic Pattern Reference

7.1. WIS2 Topic Structure

{origin|cache}/a/wis2/{centre-id}/data/{data-policy}/{earth-system-domain}/{category}/...

  • origin - Original data from the data provider

  • cache - Cached copy from a Global Cache (recommended for reliability)

  • {centre-id} - Country/organization code, prepended with ISO2C country code (e.g., de-dwd)

  • {data-policy} - WMO Unified Data Policy: core or recommended

  • {earth-system-domain} - Earth system domain from the WMO Unified Data Policy

7.2. Wildcards

Pattern Symbol Description

Single-level

+

Matches exactly one level

Multi-level

#

Matches zero or more levels (must be at end)

7.3. Examples

Pattern Matches

cache/a/wis2/de-dwd/data/#

All data from Deutscher Wetterdienst

cache/a/wis2/+/data/core/weather/surface-based-observations/#

Surface observations from all centres

cache/a/wis2/+/data/core/#

All core data from any centre

cache/a/wis2//data//weather/#

All weather data (any policy) from any centre