Server 2.4.0, app 2.11.1: ABRP forwarding removed from the server

The car sends live data to ABRP itself, and the car and the server go
over the same mobile connection: when the car cannot reach ABRP, it
cannot reach the server either, so the server's fallback bought
nothing. The forwarder, its ABRP_* settings and abrpSentUtc are gone;
the server only stores. The app stops sending abrpSentUtc (the server
ignores unknown fields, so 2.11.0 keeps working) and its texts no longer
promise a server fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QLJfZQZeAKAwSqViTmLzki
This commit is contained in:
2026-10-18 12:49:46 +02:00
co-authored by Claude Opus 5.5
parent ca2f899403
commit 9f184362f0
18 changed files with 70 additions and 359 deletions
+12 -15
View File
@@ -1,9 +1,9 @@
# Nils' MG4 # Nils' MG4
An Android Automotive app for the **MG4 (SAIC eh32)** head unit, plus a small server: the car An Android Automotive app for the **MG4 (SAIC eh32)** head unit, plus a small server: the car
sends its telemetry to my own server at car.k8s.stinnesbeck.com, which stores it in InfluxDB 2, sends its telemetry to my own server at car.k8s.stinnesbeck.com, which stores it in InfluxDB 2
forwards live data to [A Better Route Planner](https://abetterrouteplanner.com) and feeds the for the Grafana dashboards, and its live data straight to
Grafana dashboards. [A Better Route Planner](https://abetterrouteplanner.com).
No OBD dongle and no phone in the loop: the app runs on the head unit and reads the car's own No OBD dongle and no phone in the loop: the app runs on the head unit and reads the car's own
APIs. APIs.
@@ -47,18 +47,16 @@ vehicle there, so every value shows as a dash).
## How it fits together ## How it fits together
``` ```
car ──live telemetry───▶ car.k8s.stinnesbeck.com (Go) ──▶ ABRP (what the car did not send) car ──live telemetry───▶ car.k8s.stinnesbeck.com (Go)
│ ──daily chunks─────▶ │ │ ──daily chunks─────▶ │
│ ──charging sessions▶ ▼ │ ──charging sessions▶ ▼
│ InfluxDB 2 ◀── Grafana │ InfluxDB 2 ◀── Grafana
└──live telemetry (optional, direct)──────────────────▶ ABRP └──live telemetry─────▶ ABRP
``` ```
With ABRP credentials set in the app (Live data tab), the car sends live data to ABRP itself, The car sends live data to ABRP itself, with the ABRP credentials set on the Live data tab, so
so ABRP keeps getting it while the server is down; the server then forwards only samples the ABRP and the server do not depend on each other. The server only stores. The car authenticates
car could not deliver. Without them in the app, the server forwards everything with its own with the server using a token of its own, obtained by pairing with a phone (below).
ABRP credentials. The car authenticates with the server using a token of its own, obtained by
pairing with a phone (below).
## The app ## The app
@@ -118,8 +116,7 @@ pointed at this repository's releases (see `SPECS.md`).
3. Tap **Pair with phone**. Scan the QR code with a phone (or open the address shown and enter 3. Tap **Pair with phone**. Scan the QR code with a phone (or open the address shown and enter
the code), and approve with the server's approval password. The car receives its token. the code), and approve with the server's approval password. The car receives its token.
4. On the **Live data** tab, turn live data on. The service starts with the car from then on. 4. On the **Live data** tab, turn live data on. The service starts with the car from then on.
5. Optional: enter your ABRP API key and user token on the same tab, so the car also sends 5. Enter your ABRP API key and user token on the same tab, so the car sends live data to ABRP.
live data straight to ABRP.
### Config file ### Config file
@@ -141,8 +138,8 @@ installation_id = only to carry over history from an install older than 2.9.0
interval_sec = 60 # 15 / 30 / 60 / 120 / 300 interval_sec = 60 # 15 / 30 / 60 / 120 / 300
boost_low_soc = true boost_low_soc = true
low_soc_percent = 20 low_soc_percent = 20
abrp_api_key = optional: your ABRP API key, to send to ABRP directly abrp_api_key = your ABRP API key
abrp_token = optional: your ABRP user token abrp_token = your ABRP user token
``` ```
After a successful import the app deletes the file, since it holds the token; credentials live After a successful import the app deletes the file, since it holds the token; credentials live
@@ -182,7 +179,7 @@ work for the plain Gradle builds.
## Server and dashboards ## Server and dashboards
- **Server** ([`server/`](server/README.md)): Go, API only. Live telemetry, daily consumption - **Server** ([`server/`](server/README.md)): Go, API only. Live telemetry, daily consumption
chunks, charging sessions, pairing; forwards live data to ABRP. Image at chunks, charging sessions, pairing. Image at
`git.stinnesbeck.com/nils/mg4-server`. `git.stinnesbeck.com/nils/mg4-server`.
- **Kubernetes** ([`k8s/`](k8s/README.md)): plain manifests for the server, InfluxDB 2 and - **Kubernetes** ([`k8s/`](k8s/README.md)): plain manifests for the server, InfluxDB 2 and
Grafana, applied by hand with `kubectl apply -k k8s/`. Secrets go in the gitignored Grafana, applied by hand with `kubectl apply -k k8s/`. Secrets go in the gitignored
+11 -7
View File
@@ -9,15 +9,16 @@ what has been changed, and what is still to do.
## Architecture: car server at car.k8s.stinnesbeck.com ## Architecture: car server at car.k8s.stinnesbeck.com
The car sends everything to our own server. The server stores it in InfluxDB 2 and forwards live The car sends everything to our own server, which stores it in InfluxDB 2, and its live data
telemetry to ABRP, so the ABRP token lives only on the server, never in the APK. Grafana reads straight to ABRP with the ABRP credentials kept (encrypted) in the app. The server does not talk
InfluxDB directly for the dashboards. to ABRP. Grafana reads InfluxDB directly for the dashboards.
``` ```
car ──live telemetry───▶ car.k8s.stinnesbeck.com (Go) ──▶ ABRP api.iternio.com/1/tlm/send car ──live telemetry───▶ car.k8s.stinnesbeck.com (Go)
──daily chunks─────▶ │ │ ──daily chunks─────▶ │
──charging sessions▶ ▼ │ ──charging sessions▶ ▼
InfluxDB 2 ◀── Grafana (car-grafana.k8s.stinnesbeck.com) │ InfluxDB 2 ◀── Grafana (car-grafana.k8s.stinnesbeck.com)
└──live telemetry─────▶ ABRP api.iternio.com/1/tlm/send
``` ```
- **Server:** `server/`, Go 1.26, API only — see `server/README.md` for endpoints, configuration - **Server:** `server/`, Go 1.26, API only — see `server/README.md` for endpoints, configuration
@@ -112,6 +113,9 @@ tyre pressures, GPS (lat, lon, elevation, heading) and the is_* flags. It does *
INGEST_TOKEN) into git and pushed it. Commit amended and force-pushed; both values regenerated in INGEST_TOKEN) into git and pushed it. Commit amended and force-pushed; both values regenerated in
the now gitignored `k8s/secrets.yaml` (since moved to `k8s/secrets/secrets.yaml`, outside the now gitignored `k8s/secrets.yaml` (since moved to `k8s/secrets/secrets.yaml`, outside
`kubectl apply -f k8s/`). Filled-in secrets live only there from now on. `kubectl apply -f k8s/`). Filled-in secrets live only there from now on.
- 2026-10-18 — App 2.11.1, server 2.4.0: ABRP forwarding removed from the server. The car and the
server use the same mobile connection, so a server fallback for ABRP bought nothing. The car no
longer sends `abrpSentUtc`; the server ignores unknown fields, so older apps keep working.
- 2026-10-18 — App 2.11.0, server 2.3.0: - 2026-10-18 — App 2.11.0, server 2.3.0:
- Live data goes to ABRP straight from the car when ABRP credentials are set in the app - Live data goes to ABRP straight from the car when ABRP credentials are set in the app
(Live data tab or config file: `abrp_api_key`, `abrp_token`), so ABRP keeps getting it (Live data tab or config file: `abrp_api_key`, `abrp_token`), so ABRP keeps getting it
+2 -2
View File
@@ -34,8 +34,8 @@ android {
applicationId "com.stinnesbeck.nilsmg4" applicationId "com.stinnesbeck.nilsmg4"
minSdk 27 minSdk 27
targetSdk 36 targetSdk 36
versionCode 73 versionCode 74
versionName "2.11.0" versionName "2.11.1"
// Nothing about the server ships in the APK: its URL and ingest token are entered in // Nothing about the server ships in the APK: its URL and ingest token are entered in
// the app's Server tab (ServerConfig), and the database is only reachable by the server. // the app's Server tab (ServerConfig), and the database is only reachable by the server.
@@ -8,9 +8,8 @@ import android.content.SharedPreferences;
* sending) and the user token (which car in which ABRP account). Kept in {@link SecurePrefs} * sending) and the user token (which car in which ABRP account). Kept in {@link SecurePrefs}
* next to the server settings, entered on the Live data tab or imported from a config file. * next to the server settings, entered on the Live data tab or imported from a config file.
* *
* With both set, the car sends to ABRP itself, so ABRP keeps getting live data when the car * The car is the only one that sends to ABRP: the server stores live data but does not forward
* server is down; the server then only forwards what the car could not deliver. Without them, * it. Without these, ABRP gets no live data.
* the server forwards as before.
*/ */
final class AbrpConfig { final class AbrpConfig {
@@ -25,8 +25,8 @@ import java.util.concurrent.TimeUnit;
/** /**
* Foreground service that polls vehicle telemetry every 15 s, integrates consumption locally, * Foreground service that polls vehicle telemetry every 15 s, integrates consumption locally,
* syncs it to the car server in 2-minute delta chunks, and sends live telemetry samples that * syncs it to the car server in 2-minute delta chunks, and sends live telemetry samples that
* the server stores and forwards to ABRP ({@link TelemetryUploader}). The car itself never * the server stores and, with ABRP credentials set, that go straight to ABRP
* talks to ABRP and holds no ABRP credentials. * ({@link TelemetryUploader}).
* *
* Always-on by design: started at boot (BootReceiver) and on app open, never gated on * Always-on by design: started at boot (BootReceiver) and on app open, never gated on
* credentials or a toggle. The only thing it needs is a network, and a missing one is * credentials or a toggle. The only thing it needs is a network, and a missing one is
@@ -19,7 +19,7 @@ import java.util.List;
import java.util.Objects; import java.util.Objects;
/** /**
* Live telemetry to the car server, which stores it and forwards the newest sample to ABRP. * Live telemetry to the car server, which stores it, and the newest sample straight to ABRP.
* *
* Called on every service tick. While the car is in use every tick becomes a sample; the * Called on every service tick. While the car is in use every tick becomes a sample; the
* queue is sent when {@link UploadCadence} says so — every tick on a state change, otherwise * queue is sent when {@link UploadCadence} says so — every tick on a state change, otherwise
@@ -193,8 +193,6 @@ final class TelemetryUploader {
JSONObject body = new JSONObject() JSONObject body = new JSONObject()
.put("installationId", config.installationId) .put("installationId", config.installationId)
.put("samples", samples); .put("samples", samples);
// What the car already delivered to ABRP itself; the server forwards only newer samples.
if (lastAbrpUtc > 0) body.put("abrpSentUtc", lastAbrpUtc);
return ServerHttp.request("POST", config.url + "/v1/telemetry", config.token, body); return ServerHttp.request("POST", config.url + "/v1/telemetry", config.token, body);
} }
@@ -209,7 +207,7 @@ final class TelemetryUploader {
/** /**
* Sends the newest queued sample straight to ABRP when ABRP credentials are set: once per * Sends the newest queued sample straight to ABRP when ABRP credentials are set: once per
* upload, only if it is newer than the last one sent and still live. It stays queued for * upload, only if it is newer than the last one sent and still live. It stays queued for
* the server either way. * the server either way; the server does not forward to ABRP.
*/ */
private void sendNewestToAbrp() { private void sendNewestToAbrp() {
AbrpConfig abrp = AbrpConfig.get(context); AbrpConfig abrp = AbrpConfig.get(context);
+3 -3
View File
@@ -122,12 +122,12 @@
<string name="hint_steering_keys">The last key reports the car sent, newest first. If double-pressing the voice key does not open the app, a photo of this list shows how the car reports it.</string> <string name="hint_steering_keys">The last key reports the car sent, newest first. If double-pressing the voice key does not open the app, a photo of this list shows how the car reports it.</string>
<string name="steering_keys_empty">No key reports received yet. Press a steering-wheel key.</string> <string name="steering_keys_empty">No key reports received yet. Press a steering-wheel key.</string>
<string name="label_abrp_upload_enabled">Send live data</string> <string name="label_abrp_upload_enabled">Send live data</string>
<string name="hint_abrp_upload_enabled">To your server, which stores it, and to ABRP: straight from the car when the ABRP credentials below are set, otherwise forwarded by the server. Consumption and charging tracking keep running when this is off.</string> <string name="hint_abrp_upload_enabled">To your server, which stores it, and straight to ABRP when the ABRP credentials below are set. Consumption and charging tracking keep running when this is off.</string>
<string name="hint_abrp_direct">With your ABRP API key and user token here, the car sends live data to ABRP itself, so ABRP keeps getting it while your server is down. The server then forwards only what the car could not send. Leave both empty to let the server forward everything.</string> <string name="hint_abrp_direct">The car sends live data to ABRP itself with your ABRP API key and user token. Without them, ABRP gets no live data; your server only stores it.</string>
<string name="hint_abrp_api_key">ABRP API key</string> <string name="hint_abrp_api_key">ABRP API key</string>
<string name="hint_abrp_token">ABRP user token</string> <string name="hint_abrp_token">ABRP user token</string>
<string name="abrp_saved">Saved: the car sends live data to ABRP directly</string> <string name="abrp_saved">Saved: the car sends live data to ABRP directly</string>
<string name="abrp_cleared">Cleared: the server forwards live data to ABRP</string> <string name="abrp_cleared">Cleared: ABRP gets no live data</string>
<string name="abrp_invalid">Enter both, without spaces, or leave both empty</string> <string name="abrp_invalid">Enter both, without spaces, or leave both empty</string>
<string name="abrp_testing">Checking with ABRP…</string> <string name="abrp_testing">Checking with ABRP…</string>
<string name="abrp_test_ok">ABRP accepted these credentials</string> <string name="abrp_test_ok">ABRP accepted these credentials</string>
+1 -1
View File
@@ -22,7 +22,7 @@ is applied automatically. Apply them yourself.
covered by Traefik's default certificate. covered by Traefik's default certificate.
2. **Create the Secrets.** `cp k8s/secrets/secrets.example.yaml k8s/secrets/secrets.yaml` (gitignored), replace 2. **Create the Secrets.** `cp k8s/secrets/secrets.example.yaml k8s/secrets/secrets.yaml` (gitignored), replace
every `CHANGE-ME` there (`openssl rand -hex 32`): `influxdb-auth`, `mg4-server` (ingest token, every `CHANGE-ME` there (`openssl rand -hex 32`): `influxdb-auth`, `mg4-server` (ingest token,
approval password for pairing, your ABRP API key and user token), `grafana-admin` and approval password for pairing), `grafana-admin` and
`grafana-influx` (see below). Keep the template itself on placeholders only. `grafana-influx` (see below). Keep the template itself on placeholders only.
3. **Registry access.** The `mg4-server` package on git.stinnesbeck.com is public, so no pull 3. **Registry access.** The `mg4-server` package on git.stinnesbeck.com is public, so no pull
secret is needed. If you make it private, add an `imagePullSecrets` entry to `server.yaml`. secret is needed. If you make it private, add an `imagePullSecrets` entry to `server.yaml`.
-5
View File
@@ -26,11 +26,6 @@ stringData:
# Approves a car on your phone when it pairs by QR code (at least 12 characters). Leave it # Approves a car on your phone when it pairs by QR code (at least 12 characters). Leave it
# empty to switch pairing off; then only the INGEST_TOKEN above works. # empty to switch pairing off; then only the INGEST_TOKEN above works.
APPROVAL_PASSWORD: CHANGE-ME-at-least-12-characters APPROVAL_PASSWORD: CHANGE-ME-at-least-12-characters
# Your ABRP credentials. Leave either empty to switch forwarding off. Keep them set even
# when the app sends to ABRP itself: the server then forwards only what the car could not.
ABRP_API_KEY: CHANGE-ME
ABRP_USER_TOKEN: CHANGE-ME
ABRP_MIN_INTERVAL_SECONDS: "5"
--- ---
apiVersion: v1 apiVersion: v1
kind: Secret kind: Secret
+5 -6
View File
@@ -1,5 +1,5 @@
# The car server (Go). One replica on purpose: the ABRP rate limit is kept in memory, and # The car server (Go). One replica on purpose: pending pairing requests and the wrong-password
# two replicas would each forward to ABRP at the full rate. # counters are kept in memory.
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
metadata: metadata:
@@ -29,8 +29,8 @@ spec:
type: RuntimeDefault type: RuntimeDefault
containers: containers:
- name: server - name: server
# 2.3.0, pinned by digest so a re-pushed tag cannot change what runs. # 2.4.0, pinned by digest so a re-pushed tag cannot change what runs.
image: git.stinnesbeck.com/nils/mg4-server@sha256:dc5eb49cd88a1fd6a8e853c6e2291f2b5d62252e8bb6daf1750e1c3698bc7e0a image: git.stinnesbeck.com/nils/mg4-server@sha256:ab243f37812efa7294efc594f65ca65f9d551161a85e7cc641afb168e226a2fe
ports: ports:
- name: http - name: http
containerPort: 8080 containerPort: 8080
@@ -49,8 +49,7 @@ spec:
secretKeyRef: secretKeyRef:
name: influxdb-auth name: influxdb-auth
key: admin-token key: admin-token
# INGEST_TOKEN, APPROVAL_PASSWORD, ABRP_API_KEY, ABRP_USER_TOKEN, # INGEST_TOKEN, APPROVAL_PASSWORD.
# ABRP_MIN_INTERVAL_SECONDS.
envFrom: envFrom:
- secretRef: - secretRef:
name: mg4-server name: mg4-server
+5 -12
View File
@@ -1,11 +1,11 @@
# Nils' MG4 — car server # Nils' MG4 — car server
A small Go service at car.k8s.stinnesbeck.com: the API the car writes to, and the ABRP A small Go service at car.k8s.stinnesbeck.com: the API the car writes to. Data lives in
forwarder. Data lives in InfluxDB 2; Grafana reads it from there for the dashboards InfluxDB 2; Grafana reads it from there for the dashboards (`../k8s/grafana/`). The car sends
(`../k8s/grafana/`). its live data to ABRP itself; the server does not talk to ABRP (removed in 2.4.0).
``` ```
car ──live telemetry───▶ server ──▶ ABRP api.iternio.com/1/tlm/send car ──live telemetry───▶ server
──daily chunks─────▶ │ ──daily chunks─────▶ │
──charging sessions▶ ▼ ──charging sessions▶ ▼
InfluxDB 2 ◀── Grafana InfluxDB 2 ◀── Grafana
@@ -20,14 +20,10 @@ Environment only; the server refuses to start when a required value is missing.
| `INFLUX_URL`, `INFLUX_TOKEN`, `INFLUX_ORG` | required; the token needs read/write on the bucket | | `INFLUX_URL`, `INFLUX_TOKEN`, `INFLUX_ORG` | required; the token needs read/write on the bucket |
| `INFLUX_BUCKET` | default `mg4` | | `INFLUX_BUCKET` | default `mg4` |
| `INGEST_TOKEN` | required, at least 24 characters: `openssl rand -hex 32` | | `INGEST_TOKEN` | required, at least 24 characters: `openssl rand -hex 32` |
| `ABRP_API_KEY`, `ABRP_USER_TOKEN` | optional; both needed to forward |
| `ABRP_MIN_INTERVAL_SECONDS` | default 5 |
| `APPROVAL_PASSWORD` | optional, at least 12 characters; switches pairing on (see below) | | `APPROVAL_PASSWORD` | optional, at least 12 characters; switches pairing on (see below) |
| `PUBLIC_URL` | the address a phone uses, for the pairing QR code; derived from the request if empty | | `PUBLIC_URL` | the address a phone uses, for the pairing QR code; derived from the request if empty |
| `LISTEN_ADDR` | default `:8080` | | `LISTEN_ADDR` | default `:8080` |
The ABRP credentials never reach the car.
## Endpoints ## Endpoints
The ingest endpoints accept `Authorization: Bearer <token>` with either the shared `INGEST_TOKEN` The ingest endpoints accept `Authorization: Bearer <token>` with either the shared `INGEST_TOKEN`
@@ -39,10 +35,7 @@ with (403 otherwise); the shared token works for any.
dropped, known ones bounded: a value out of range is dropped on its own, and a sample without dropped, known ones bounded: a value out of range is dropped on its own, and a sample without
a plausible time (before 2001, or more than an hour ahead) is dropped whole. The rest is stored a plausible time (before 2001, or more than an hour ahead) is dropped whole. The rest is stored
and the answer is `200 {stored, droppedCount, dropped}`, even when nothing was usable, since and the answer is `200 {stored, droppedCount, dropped}`, even when nothing was usable, since
the car would only send it again. The newest sample is forwarded to ABRP if it is under the car would only send it again.
5 minutes old and not ahead of the server's clock. A car that sends to ABRP itself adds
`abrpSentUtc`, the newest sample ABRP took from it; only samples newer than that are
forwarded, so ABRP gets each one once.
- `PUT /v1/consumption/daily` — one delta chunk of a day's consumption. - `PUT /v1/consumption/daily` — one delta chunk of a day's consumption.
- `PUT /v1/charging-sessions/{sessionId}` — a finished DC session with its curve. - `PUT /v1/charging-sessions/{sessionId}` — a finished DC session with its curve.
- `GET /v1/consumption/history/{installationId}` — the car reads its own totals back. - `GET /v1/consumption/history/{installationId}` — the car reads its own totals back.
-142
View File
@@ -1,142 +0,0 @@
package main
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"net/url"
"strings"
"sync"
"time"
)
// Forwarder sends live telemetry on to A Better Route Planner.
//
// The ABRP credentials exist only here, from ABRP_API_KEY and ABRP_USER_TOKEN in the
// server's environment; the car never holds them. Forwarding is off when either is empty.
//
// Only the newest sample of a batch is sent, and at most one per MinInterval. A failed
// forward is logged and dropped, never queued: ABRP plans from the car's state now, and a
// backlog replayed later would describe a car that has moved on. The stored copy in
// InfluxDB is unaffected either way.
type Forwarder struct {
APIKey string
UserToken string
MinInterval time.Duration
Endpoint string
Client *http.Client
mu sync.Mutex
lastUTC int64
lastSentAt time.Time
}
const (
abrpEndpoint = "https://api.iternio.com/1/tlm/send"
// Older than this, a sample is history rather than live state, and ABRP gets nothing.
abrpMaxAge = 5 * time.Minute
// Newer than now by more than this, a sample's clock is wrong, and ABRP gets nothing.
abrpMaxAhead = 30 * time.Second
)
func (f *Forwarder) configured() bool {
return f != nil && f.APIKey != "" && f.UserToken != ""
}
// pick chooses the sample to forward, or false when there is nothing new or it is too
// soon. It records the choice, so two batches arriving together cannot both forward.
func (f *Forwarder) pick(samples []TelemetrySample, now time.Time) (TelemetrySample, bool) {
// A sample stamped ahead of now has a wrong clock. Taking it would also set lastUTC into
// the future, and every real sample after it would look older and never be sent.
latest := now.Add(abrpMaxAhead).Unix()
var newest TelemetrySample
found := false
for _, s := range samples {
if s.UTC <= latest && (!found || s.UTC > newest.UTC) {
newest, found = s, true
}
}
if !found || now.Sub(time.Unix(newest.UTC, 0)) > abrpMaxAge {
return TelemetrySample{}, false
}
f.mu.Lock()
defer f.mu.Unlock()
if f.lastUTC != 0 && newest.UTC <= f.lastUTC {
return TelemetrySample{}, false
}
if !f.lastSentAt.IsZero() && now.Sub(f.lastSentAt) < f.MinInterval {
return TelemetrySample{}, false
}
f.lastUTC = newest.UTC
f.lastSentAt = now
return newest, true
}
// Forward is fire-and-forget: the car's request is answered without waiting for ABRP.
func (f *Forwarder) Forward(samples []TelemetrySample) {
if !f.configured() {
return
}
sample, ok := f.pick(samples, time.Now())
if !ok {
return
}
go func() {
if err := f.send(context.Background(), sample); err != nil {
slog.Warn("ABRP forward failed", "error", err)
}
}()
}
// tlm is the object ABRP expects: the same field names, flags as 0/1.
func tlm(sample TelemetrySample) map[string]any {
out := map[string]any{"utc": sample.UTC}
for field, value := range sample.Numbers {
out[field] = value
}
for field, value := range sample.Flags {
if value {
out[field] = 1
} else {
out[field] = 0
}
}
return out
}
func (f *Forwarder) send(ctx context.Context, sample TelemetrySample) error {
payload, err := json.Marshal(tlm(sample))
if err != nil {
return err
}
// Credentials go in the form body, never the URL, which ends up in access logs.
form := url.Values{"api_key": {f.APIKey}, "token": {f.UserToken}, "tlm": {string(payload)}}
endpoint := f.Endpoint
if endpoint == "" {
endpoint = abrpEndpoint
}
ctx, cancel := context.WithTimeout(ctx, 8*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, strings.NewReader(form.Encode()))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
client := f.Client
if client == nil {
client = http.DefaultClient
}
resp, err := client.Do(req)
if err != nil {
// A url.Error from the client quotes the URL, which holds no credentials.
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
// The response body can echo the request; only the status is reported.
return fmt.Errorf("rejected: HTTP %d", resp.StatusCode)
}
return nil
}
+3 -21
View File
@@ -13,7 +13,7 @@ import (
// API serves the car. Every endpoint but /v1/health needs the ingest token. // API serves the car. Every endpoint but /v1/health needs the ingest token.
// //
// POST /v1/telemetry live samples; stored, newest forwarded to ABRP // POST /v1/telemetry live samples, stored
// PUT /v1/consumption/daily one delta chunk of a day's consumption // PUT /v1/consumption/daily one delta chunk of a day's consumption
// PUT /v1/charging-sessions/{sessionId} one finished DC charging session // PUT /v1/charging-sessions/{sessionId} one finished DC charging session
// GET /v1/consumption/history/{installationId} the car reads its own totals back // GET /v1/consumption/history/{installationId} the car reads its own totals back
@@ -24,7 +24,6 @@ import (
type API struct { type API struct {
Store Store Store Store
IngestToken string IngestToken string
Forwarder *Forwarder
Pairing *Pairing Pairing *Pairing
// PublicURL is the server's address as a phone reaches it, for the pairing QR code; // PublicURL is the server's address as a phone reaches it, for the pairing QR code;
// derived from the request when empty. // derived from the request when empty.
@@ -117,27 +116,11 @@ func (a *API) ingest(h handler) http.HandlerFunc {
} }
} }
// notYetAtAbrp leaves out the samples the car already sent to ABRP itself.
func notYetAtAbrp(samples []TelemetrySample, sentUTC int64) []TelemetrySample {
if sentUTC == 0 {
return samples
}
out := make([]TelemetrySample, 0, len(samples))
for _, s := range samples {
if s.UTC > sentUTC {
out = append(out, s)
}
}
return out
}
// maxDroppedListed caps the dropped values named in a telemetry response and log line. // maxDroppedListed caps the dropped values named in a telemetry response and log line.
const maxDroppedListed = 20 const maxDroppedListed = 20
// telemetry stores the batch first; only once the write succeeded is the newest sample // telemetry stores the batch. The answer says how many samples were stored and what was
// handed to ABRP, so the car's retry of a failed request cannot leave ABRP with data the // dropped; a batch with nothing usable is still a 200, since sending it again would not help.
// database never got. The answer says how many samples were stored and what was dropped;
// a batch with nothing usable is still a 200, since sending it again would not help.
func (a *API) telemetry(w http.ResponseWriter, r *http.Request, c caller, body []byte) error { func (a *API) telemetry(w http.ResponseWriter, r *http.Request, c caller, body []byte) error {
batch, err := validateTelemetryBatch(body, time.Now()) batch, err := validateTelemetryBatch(body, time.Now())
if err != nil { if err != nil {
@@ -150,7 +133,6 @@ func (a *API) telemetry(w http.ResponseWriter, r *http.Request, c caller, body [
if err := a.Store.WriteTelemetry(r.Context(), batch.InstallationID, batch.Samples); err != nil { if err := a.Store.WriteTelemetry(r.Context(), batch.InstallationID, batch.Samples); err != nil {
return err return err
} }
a.Forwarder.Forward(notYetAtAbrp(batch.Samples, batch.AbrpSentUTC))
} }
dropped := batch.Dropped dropped := batch.Dropped
if len(dropped) > maxDroppedListed { if len(dropped) > maxDroppedListed {
+1 -21
View File
@@ -76,7 +76,7 @@ const (
func newTestAPI() (*API, *fakeStore) { func newTestAPI() (*API, *fakeStore) {
store := &fakeStore{} store := &fakeStore{}
return &API{Store: store, IngestToken: testToken, Forwarder: &Forwarder{}, return &API{Store: store, IngestToken: testToken,
Pairing: NewPairing(testApprovalPassword, nil), PublicURL: "https://car.example.com"}, store Pairing: NewPairing(testApprovalPassword, nil), PublicURL: "https://car.example.com"}, store
} }
@@ -204,23 +204,3 @@ func jsonInt(v int64) string {
b, _ := json.Marshal(v) b, _ := json.Marshal(v)
return string(b) return string(b)
} }
func TestSamplesTheCarSentToAbrpAreNotForwardedAgain(t *testing.T) {
samples := []TelemetrySample{{UTC: 100}, {UTC: 115}, {UTC: 130}}
if got := notYetAtAbrp(samples, 0); len(got) != 3 {
t.Fatalf("without abrpSentUtc everything is forwardable: %d", len(got))
}
if got := notYetAtAbrp(samples, 130); len(got) != 0 {
t.Fatalf("the car sent the newest: %d left", len(got))
}
if got := notYetAtAbrp(samples, 115); len(got) != 1 || got[0].UTC != 130 {
t.Fatalf("only the newer sample: %+v", got)
}
b, err := validateTelemetryBatch([]byte(`{"installationId":"`+testInstallation+`","abrpSentUtc":1790000000,"samples":[{"utc":1790000000}]}`), testNow)
if err != nil || b.AbrpSentUTC != 1790000000 {
t.Fatalf("abrpSentUtc not read: %v %d", err, b.AbrpSentUTC)
}
if _, err := validateTelemetryBatch([]byte(`{"installationId":"`+testInstallation+`","abrpSentUtc":"x","samples":[{"utc":1790000000}]}`), testNow); !isValidationError(err) {
t.Fatal("a malformed abrpSentUtc was accepted")
}
}
+1 -67
View File
@@ -1,10 +1,6 @@
package main package main
import ( import (
"io"
"net/http"
"net/http/httptest"
"net/url"
"testing" "testing"
"time" "time"
) )
@@ -49,68 +45,6 @@ func TestAggregateDaysNewestSohWins(t *testing.T) {
} }
} }
func TestTlmSendsFlagsAsZeroOrOne(t *testing.T) {
got := tlm(TelemetrySample{UTC: testNowS, Numbers: map[string]float64{"soc": 50},
Flags: map[string]bool{"is_charging": true, "is_parked": false}})
if got["utc"] != int64(testNowS) || got["soc"] != 50.0 || got["is_charging"] != 1 || got["is_parked"] != 0 {
t.Fatalf("tlm %+v", got)
}
}
func TestForwarderPicksOnlyTheNewestFreshSampleOncePerInterval(t *testing.T) {
f := &Forwarder{MinInterval: 5 * time.Second}
now := time.Unix(testNowS, 0)
got, ok := f.pick([]TelemetrySample{{UTC: testNowS - 10}, {UTC: testNowS - 2}, {UTC: testNowS - 5}}, now)
if !ok || got.UTC != testNowS-2 {
t.Fatalf("picked %+v %v", got, ok)
}
if _, ok := f.pick([]TelemetrySample{{UTC: testNowS}}, now.Add(time.Second)); ok {
t.Fatal("forwarded again within the interval")
}
if _, ok := f.pick([]TelemetrySample{{UTC: testNowS - 3}}, now.Add(time.Minute)); ok {
t.Fatal("forwarded an older sample than the last one")
}
// A sample from ten minutes ago describes a car that has moved on.
fresh := &Forwarder{MinInterval: 5 * time.Second}
if _, ok := fresh.pick([]TelemetrySample{{UTC: testNowS - 600}}, now); ok {
t.Fatal("forwarded a stale sample")
}
// A sample from a clock a day ahead is skipped, and does not block the real ones after it.
ahead := &Forwarder{MinInterval: 5 * time.Second}
got, ok = ahead.pick([]TelemetrySample{{UTC: testNowS + 86400}, {UTC: testNowS - 1}}, now)
if !ok || got.UTC != testNowS-1 {
t.Fatalf("picked %+v %v", got, ok)
}
if _, ok := ahead.pick([]TelemetrySample{{UTC: testNowS + 86400}}, now.Add(time.Minute)); ok {
t.Fatal("forwarded a sample from the future")
}
if _, ok := ahead.pick([]TelemetrySample{{UTC: testNowS + 60}}, now.Add(time.Minute)); !ok {
t.Fatal("a real sample after a future one was not forwarded")
}
}
func TestForwarderPutsCredentialsInTheBodyNotTheURL(t *testing.T) {
received := make(chan *http.Request, 1)
var form url.Values
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
form, _ = url.ParseQuery(string(body))
received <- r
}))
defer srv.Close()
f := &Forwarder{APIKey: "key-123", UserToken: "token-456", Endpoint: srv.URL + "/1/tlm/send"}
if err := f.send(t.Context(), TelemetrySample{UTC: testNowS, Numbers: map[string]float64{"soc": 80}}); err != nil {
t.Fatal(err)
}
r := <-received
if r.URL.RawQuery != "" {
t.Fatalf("credentials leaked into the URL: %s", r.URL.RawQuery)
}
if form.Get("api_key") != "key-123" || form.Get("token") != "token-456" || form.Get("tlm") != `{"soc":80,"utc":1790000000}` {
t.Fatalf("form %v", form)
}
}
func TestLoadConfigFailsClosed(t *testing.T) { func TestLoadConfigFailsClosed(t *testing.T) {
env := map[string]string{ env := map[string]string{
"INFLUX_URL": "http://influxdb:8086", "INFLUX_TOKEN": "t", "INFLUX_ORG": "mg4", "INFLUX_URL": "http://influxdb:8086", "INFLUX_TOKEN": "t", "INFLUX_ORG": "mg4",
@@ -118,7 +52,7 @@ func TestLoadConfigFailsClosed(t *testing.T) {
} }
get := func(k string) string { return env[k] } get := func(k string) string { return env[k] }
c, err := loadConfig(get) c, err := loadConfig(get)
if err != nil || c.listenAddr != ":8080" || c.influxBucket != "mg4" || c.abrpMinInterval != 5*time.Second { if err != nil || c.listenAddr != ":8080" || c.influxBucket != "mg4" {
t.Fatalf("defaults: %+v %v", c, err) t.Fatalf("defaults: %+v %v", c, err)
} }
env["INGEST_TOKEN"] = "short" env["INGEST_TOKEN"] = "short"
+20 -40
View File
@@ -1,5 +1,6 @@
// Command server is the Nils' MG4 car server: the API the car writes its telemetry, // Command server is the Nils' MG4 car server: the API the car writes its telemetry,
// consumption and charging sessions to, stored in InfluxDB 2 and forwarded live to ABRP. // consumption and charging sessions to, stored in InfluxDB 2. The car sends live data to ABRP
// itself.
// Dashboards are Grafana's job; Grafana reads the same bucket directly. // Dashboards are Grafana's job; Grafana reads the same bucket directly.
package main package main
@@ -11,7 +12,6 @@ import (
"net/http" "net/http"
"os" "os"
"os/signal" "os/signal"
"strconv"
"syscall" "syscall"
"time" "time"
) )
@@ -22,33 +22,28 @@ const (
) )
type config struct { type config struct {
listenAddr string listenAddr string
influxURL string influxURL string
influxToken string influxToken string
influxOrg string influxOrg string
influxBucket string influxBucket string
ingestToken string ingestToken string
abrpAPIKey string approvalPass string
abrpUserToken string publicURL string
abrpMinInterval time.Duration
approvalPass string
publicURL string
} }
// loadConfig reads the environment and fails closed: a missing InfluxDB setting or a // loadConfig reads the environment and fails closed: a missing InfluxDB setting or a
// short ingest token stops the server from starting rather than serving something broken. // short ingest token stops the server from starting rather than serving something broken.
func loadConfig(getenv func(string) string) (config, error) { func loadConfig(getenv func(string) string) (config, error) {
c := config{ c := config{
listenAddr: getenv("LISTEN_ADDR"), listenAddr: getenv("LISTEN_ADDR"),
influxURL: getenv("INFLUX_URL"), influxURL: getenv("INFLUX_URL"),
influxToken: getenv("INFLUX_TOKEN"), influxToken: getenv("INFLUX_TOKEN"),
influxOrg: getenv("INFLUX_ORG"), influxOrg: getenv("INFLUX_ORG"),
influxBucket: getenv("INFLUX_BUCKET"), influxBucket: getenv("INFLUX_BUCKET"),
ingestToken: getenv("INGEST_TOKEN"), ingestToken: getenv("INGEST_TOKEN"),
abrpAPIKey: getenv("ABRP_API_KEY"), approvalPass: getenv("APPROVAL_PASSWORD"),
abrpUserToken: getenv("ABRP_USER_TOKEN"), publicURL: getenv("PUBLIC_URL"),
approvalPass: getenv("APPROVAL_PASSWORD"),
publicURL: getenv("PUBLIC_URL"),
} }
if c.listenAddr == "" { if c.listenAddr == "" {
c.listenAddr = ":8080" c.listenAddr = ":8080"
@@ -69,14 +64,6 @@ func loadConfig(getenv func(string) string) (config, error) {
if c.approvalPass != "" && len(c.approvalPass) < minApprovalPasswordLength { if c.approvalPass != "" && len(c.approvalPass) < minApprovalPasswordLength {
return config{}, fmt.Errorf("APPROVAL_PASSWORD must be at least %d characters", minApprovalPasswordLength) return config{}, fmt.Errorf("APPROVAL_PASSWORD must be at least %d characters", minApprovalPasswordLength)
} }
c.abrpMinInterval = 5 * time.Second
if raw := getenv("ABRP_MIN_INTERVAL_SECONDS"); raw != "" {
seconds, err := strconv.ParseFloat(raw, 64)
if err != nil || seconds < 0 {
return config{}, errors.New("ABRP_MIN_INTERVAL_SECONDS must be a non-negative number")
}
c.abrpMinInterval = time.Duration(seconds * float64(time.Second))
}
return c, nil return c, nil
} }
@@ -90,12 +77,6 @@ func main() {
store := NewInfluxStore(cfg.influxURL, cfg.influxToken, cfg.influxOrg, cfg.influxBucket) store := NewInfluxStore(cfg.influxURL, cfg.influxToken, cfg.influxOrg, cfg.influxBucket)
defer store.Close() defer store.Close()
forwarder := &Forwarder{
APIKey: cfg.abrpAPIKey,
UserToken: cfg.abrpUserToken,
MinInterval: cfg.abrpMinInterval,
Client: &http.Client{Timeout: 10 * time.Second},
}
// Paired cars' tokens must be known before the first request; starting without them // Paired cars' tokens must be known before the first request; starting without them
// would answer every paired car with 401, so a failed load stops the start instead. // would answer every paired car with 401, so a failed load stops the start instead.
loadCtx, cancelLoad := context.WithTimeout(context.Background(), 30*time.Second) loadCtx, cancelLoad := context.WithTimeout(context.Background(), 30*time.Second)
@@ -106,8 +87,7 @@ func main() {
os.Exit(1) os.Exit(1)
} }
pairing := NewPairing(cfg.approvalPass, devices) pairing := NewPairing(cfg.approvalPass, devices)
api := &API{Store: store, IngestToken: cfg.ingestToken, Forwarder: forwarder, api := &API{Store: store, IngestToken: cfg.ingestToken, Pairing: pairing, PublicURL: cfg.publicURL}
Pairing: pairing, PublicURL: cfg.publicURL}
srv := &http.Server{ srv := &http.Server{
Addr: cfg.listenAddr, Addr: cfg.listenAddr,
@@ -122,7 +102,7 @@ func main() {
defer stop() defer stop()
go func() { go func() {
slog.Info("listening", "addr", cfg.listenAddr, "bucket", cfg.influxBucket, slog.Info("listening", "addr", cfg.listenAddr, "bucket", cfg.influxBucket,
"abrp", forwarder.configured(), "pairing", pairing.Enabled(), "pairedDevices", len(devices)) "pairing", pairing.Enabled(), "pairedDevices", len(devices))
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
slog.Error("server failed", "error", err) slog.Error("server failed", "error", err)
os.Exit(1) os.Exit(1)
-3
View File
@@ -107,7 +107,4 @@ type TelemetryBatch struct {
Samples []TelemetrySample Samples []TelemetrySample
// Dropped names the values and samples validation left out, e.g. "samples[3].speed". // Dropped names the values and samples validation left out, e.g. "samples[3].speed".
Dropped []string Dropped []string
// AbrpSentUTC is the newest sample the car delivered to ABRP itself (0: none). The
// server forwards only samples newer than that, so ABRP gets each one once.
AbrpSentUTC int64
} }
-5
View File
@@ -273,11 +273,6 @@ func validateTelemetryBatch(body []byte, now time.Time) (TelemetryBatch, error)
return TelemetryBatch{}, invalid("samples is invalid") return TelemetryBatch{}, invalid("samples is invalid")
} }
batch := TelemetryBatch{InstallationID: id, Samples: make([]TelemetrySample, 0, len(raw))} batch := TelemetryBatch{InstallationID: id, Samples: make([]TelemetrySample, 0, len(raw))}
if o.has("abrpSentUtc") {
if batch.AbrpSentUTC, err = integer(o["abrpSentUtc"], "abrpSentUtc"); err != nil {
return TelemetryBatch{}, err
}
}
latest := now.Add(maxClockAhead).Unix() latest := now.Add(maxClockAhead).Unix()
for i, entry := range raw { for i, entry := range raw {
name := fmt.Sprintf("samples[%d]", i) name := fmt.Sprintf("samples[%d]", i)