Telemetry API

A C++17 telemetry ingestion service for accepting IoT sensor readings, storing recent readings in memory, emitting structured telemetry events, and exposing runtime metrics.

The project combines a small REST API with a telemetry pipeline: incoming readings update metrics, generate structured events, flow through a bounded asynchronous queue, and can be exported to stdout, files, HTTP endpoints, TCP, or UDP sinks.

🚀 Live Deployment

The Telemetry API is deployed on Render using a multi-stage Docker build with automatic health checks. Runtime metrics are exposed through a protected Prometheus-compatible /metrics endpoint and can be scraped and visualized by external monitoring systems such as Prometheus and Grafana Cloud.

Live Links

• API Health Endpoint: https://telemetry-api-s3b2.onrender.com/health
• Grafana Dashboard: https://jumbopeach1255.grafana.net/public-dashboards/5eb8f1deb72d4fa6b1ed6b0e22ae938e

Note

• The root endpoint (/) is intentionally not implemented. Use the /health endpoint to verify the service is running.
• The /metrics endpoint is protected using Bearer authentication via TELEMETRY_METRICS_TOKEN and is intended to be scraped by Prometheus-compatible monitoring systems such as Grafana Cloud.

External Grafana Visualization

The /metrics endpoint exposes Prometheus-compatible metrics that can be scraped by an external Prometheus-compatible monitoring setup and visualized in Grafana.

Features

• REST API built with C++17 and cpp-httplib
• JSON request and response handling with nlohmann/json
• Thread-safe in-memory telemetry storage
• Average temperature and reading count statistics
• Structured event logging with correlation and request IDs
• Asynchronous telemetry pipeline with batching, sampling, rate limiting, and overflow policies
• Bounded per-sink queues with retry and exponential backoff
• Configurable sink implementations for stdout, file, HTTP, TCP, and UDP
• Counters, gauges, histograms, timers, and summaries
• Prometheus-compatible metrics endpoint
• JSON metrics endpoint through the exporter abstraction
• Health endpoint with process diagnostics
• Server-sent event stream for live telemetry events
• Threshold alert engine with cooldowns and stdout notifications
• JSON configuration with environment variable overrides
• Config file watcher for selected runtime pipeline settings
• Unit tests for storage, configuration, metrics, pipeline behavior, serialization, and sink failures
• Optional clang-tidy, cppcheck, and coverage build support

Tech Stack

• C++17
• CMake 3.16+
• cpp-httplib for the embedded HTTP server
• nlohmann/json for JSON parsing and serialization
• GitHub Actions for CI

Both third-party headers are vendored under include/, so no package manager is required for the default build.

Project Layout

telemetry-api/
|-- include/
|   |-- httplib.h
|   |-- nlohmann/json.hpp
|   |-- telemetry.hpp
|   |-- telemetry_alerting.hpp
|   |-- telemetry_config.hpp
|   |-- telemetry_diagnostics.hpp
|   |-- telemetry_event.hpp
|   |-- telemetry_exporter.hpp
|   |-- telemetry_metrics.hpp
|   |-- telemetry_monitoring.hpp
|   |-- telemetry_pipeline.hpp
|   |-- telemetry_queue.hpp
|   |-- telemetry_serializer.hpp
|   |-- telemetry_sinks.hpp
|   `-- telemetry_store.hpp
|-- src/
|   |-- main.cpp
|   `-- telemetry_*.cpp
|-- tests/
|   |-- metrics_test.cpp
|   |-- pipeline_test.cpp
|   |-- serialization_test.cpp
|   |-- sink_failure_test.cpp
|   `-- telemetry_test.cpp
|-- .github/workflows/
|-- CMakeLists.txt
`-- README.md

Build

cmake -S . -B build
cmake --build build --parallel

On Windows with a multi-config generator, the executable is usually placed under a configuration directory such as build/Debug/telemetry_api.exe or build/Release/telemetry_api.exe.

Run

Linux/macOS:

./build/telemetry_api

Windows PowerShell:

.\build\Debug\telemetry_api.exe

By default, the server listens on:

http://0.0.0.0:8080

Use TELEMETRY_PORT or a JSON config file to change the port.

Set TELEMETRY_METRICS_TOKEN to require Bearer authentication on /metrics. Grafana Cloud Hosted Collector requires the metrics endpoint to be authenticated.

Deploy on Render

The repository includes a multi-stage Dockerfile and a render.yaml Blueprint.

1. Push the repository to GitHub.
2. In Render, create a new Blueprint and connect the repository.
3. Render builds the Docker image and checks /health before making the service live.

The server reads Render's PORT environment variable automatically. TELEMETRY_PORT remains available as an explicit override.

After deployment, verify https://telemetry-api-s3b2.onrender.com/health.

API

POST /telemetry

Accepts a sensor reading, stores it in memory, updates sensor metrics, and emits a structured event into the telemetry pipeline.

curl -X POST http://localhost:8080/telemetry \
  -H "Content-Type: application/json" \
  -H "X-Correlation-Id: demo-correlation-id" \
  -H "X-Request-Id: demo-request-id" \
  -d '{"sensor_id":"sensor_1","temperature":25.5,"humidity":60,"timestamp":1710000000}'

Successful response:

Telemetry added

Invalid JSON or missing fields return 400:

{ "error": "invalid telemetry payload" }

GET /telemetry

Returns all telemetry records currently stored in memory.

curl http://localhost:8080/telemetry

[
  {
    "sensor_id": "sensor_1",
    "temperature": 25.5,
    "humidity": 60,
    "timestamp": 1710000000
  }
]

GET /stats

Returns aggregate statistics for stored readings.

curl http://localhost:8080/stats

{ "avg_temperature": 25.5, "count": 1 }

POST /flush

Flushes the telemetry pipeline.

curl -X POST http://localhost:8080/flush

flushed

GET /health

Returns process health and lightweight runtime diagnostics.

curl http://localhost:8080/health

Example fields include status, process_id, host, thread_id, and uptime_ms.

GET /metrics

Returns a Prometheus-compatible metrics scrape.

curl http://localhost:8080/metrics

Metrics include HTTP request timing, active requests, pipeline counters, sink drops, and sensor reading metrics.

GET /metrics/json

Returns application metrics as schema-tagged JSON through the exporter abstraction.

curl http://localhost:8080/metrics/json

GET /telemetry/live

Streams accepted telemetry events as server-sent events.

curl -N http://localhost:8080/telemetry/live

The stream sends heartbeat comments when no events are available.

Configuration

The server can run with defaults, environment variables, or a JSON config file.

Set TELEMETRY_CONFIG to load a JSON configuration file:

$env:TELEMETRY_CONFIG = "telemetry.config.json"
.\build\Debug\telemetry_api.exe

Example telemetry.config.json:

{
  "pipeline": {
    "queue_size": 8192,
    "batch_size": 128,
    "worker_count": 2,
    "flush_interval_ms": 1000,
    "overflow_policy": "drop_newest",
    "sampling_rate": 1.0,
    "rate_limit_per_second": 0
  },
  "sinks": [
    {
      "type": "stdout",
      "name": "console"
    },
    {
      "type": "file",
      "name": "telemetry-file",
      "target": "telemetry.log"
    },
    {
      "type": "http",
      "name": "http-demo",
      "target": "http://localhost:9000/events"
    },
    {
      "type": "tcp",
      "name": "tcp-demo",
      "target": "tcp://localhost:9001"
    },
    {
      "type": "udp",
      "name": "udp-demo",
      "target": "udp://localhost:9002"
    }
  ],
  "alerts": [
    {
      "name": "high-temperature",
      "metric": "sensor_temperature_celsius",
      "op": ">",
      "threshold": 80,
      "cooldown_ms": 30000
    }
  ]
}

Using Sinks

• stdout and file sinks: These sinks can work immediately without any external receiver.
• HTTP, TCP, and UDP sinks: These network sinks require a reachable downstream receiver listening at the specified destination for end-to-end delivery.
• Localhost targets: Using localhost as a target (like in the example above) is only suitable when the external receiver is running on the very same machine or container as the telemetry service.

Supported sink type values:

• stdout
• file
• http
• tcp
• udp

For network sinks (http, tcp, and udp), the configured target must be reachable from the telemetry service and must have a compatible receiver listening at the specified destination.

Supported overflow policies:

• drop_newest
• drop_oldest
• block

Supported alert operators:

• >
• >=
• <
• <=
• ==

Environment variables override file values for selected runtime settings:

Variable	Description
TELEMETRY_CONFIG	Path to a JSON config file
TELEMETRY_PORT	HTTP server port
TELEMETRY_QUEUE_SIZE	Main pipeline queue size
TELEMETRY_BATCH_SIZE	Pipeline batch size
TELEMETRY_WORKERS	Number of pipeline worker threads
TELEMETRY_SAMPLING_RATE	Event sampling rate from 0.0 to 1.0

When TELEMETRY_CONFIG is set, the configuration file can be watched for runtime updates to supported pipeline settings. Sink and alert configuration is loaded during startup.

Tests

Build and run the test suite:

cmake -S . -B build
cmake --build build --parallel
ctest --test-dir build --output-on-failure

The configured tests are:

• telemetry_test
• config_test
• metrics_test
• pipeline_test
• serialization_test
• sink_failure_test

Tooling

Enable clang-tidy during compilation:

cmake -S . -B build -DTELEMETRY_ENABLE_CLANG_TIDY=ON
cmake --build build --parallel

Run cppcheck if it is installed:

cmake --build build --target cppcheck

Build with coverage instrumentation on GCC or Clang:

cmake -S . -B build-coverage -DTELEMETRY_ENABLE_COVERAGE=ON
cmake --build build-coverage --parallel
ctest --test-dir build-coverage --output-on-failure

CI

GitHub Actions builds the project, runs tests, executes cppcheck, and validates a coverage build on Ubuntu.

License

This project is for educational purposes and demonstration of modern C++ backend and telemetry service design.