Showing 10 changed files with 262 additions and 0 deletions
+2 -0
.doc/database/tables/data_workers.md
@@ -33,6 +33,8 @@ Referenced by:
33 33
 - `dhcp-router`, type `dhcp`
34 34
 - `mdns-listener`, type `mdns`
35 35
 
36
+`dhcp-router` este actualizat de endpoint-ul `POST /api/collect/dhcp-leases` când MikroTik împinge evenimente de lease de pe `192.168.2.1`.
37
+
36 38
 ## Definition
37 39
 
38 40
 ```sql
+2 -0
.doc/database/tables/dhcp_leases.md
@@ -2,6 +2,8 @@
2 2
 
3 3
 Stores observed DHCP lease/reservation data.
4 4
 
5
+Rows are populated by the DHCP push collector endpoint `POST /api/collect/dhcp-leases`, normally called from the MikroTik `lease-script` on `192.168.2.1`.
6
+
5 7
 ## Columns
6 8
 
7 9
 | Column | Type | Null | Default | Notes |
+1 -0
.doc/development-log.md
@@ -38,3 +38,4 @@ Jurnalele explica deciziile care schimba scopul aplicatiei, regulile de operare,
38 38
 | 2026-06-08 | Commit and Push Is the Development Contract | [Deployment and operations](development-logs/deployment-operations.md#2026-06-08---commit-and-push-is-the-development-contract) |
39 39
 | 2026-06-09 | SQLite Runtime Source of Truth | [Database](development-logs/database.md#2026-06-09---sqlite-runtime-source-of-truth) |
40 40
 | 2026-06-09 | Relational Runtime Schema | [Database](development-logs/database.md#2026-06-09---relational-runtime-schema) |
41
+| 2026-06-10 | DHCP Lease Push Collector | [Database](development-logs/database.md#2026-06-10---dhcp-lease-push-collector) |
+16 -0
.doc/development-logs/database.md
@@ -41,3 +41,19 @@ Scop:
41 41
 - schema sa poata sustine inventar, DNS, vhosturi, observatii externe si certificate fara sa piarda istoric operational
42 42
 - aliasurile/vhosturile retrase sa ramana audibile in baza de date
43 43
 - structura sa fie extensibila fara sa supraincarce tabelul `hosts`
44
+
45
+## 2026-06-10 - DHCP Lease Push Collector
46
+
47
+Observatie: DHCP-ul de pe `192.168.2.1` este autoritatea pentru alocarea IP-urilor LAN, dar baza runtime avea doar tabela `dhcp_leases`, fara un colector operational.
48
+
49
+Decizie:
50
+
51
+- MikroTik impinge evenimentele de lease prin `lease-script` catre `POST /api/collect/dhcp-leases`
52
+- endpoint-ul este protejat cu `HOST_MANAGER_DHCP_PUSH_TOKEN`, separat de sesiunea OTP pentru UI
53
+- ingest-ul scrie doar in `dhcp_leases` si actualizeaza workerul `dhcp-router`
54
+- observatiile DHCP raman material de audit/propunere si nu modifica automat host registry-ul sau manifestul DNS
55
+
56
+Scop:
57
+
58
+- sa colectam lease-uri fara credential SSH pentru router in aplicatie
59
+- sa pastram DHCP ca sursa observata cu prioritate mare, dar fara side effect automat asupra DNS-ului local
+1 -0
.doc/host-manager.md
@@ -28,6 +28,7 @@ Endpoint-uri publice:
28 28
 - `/api/session` — status boolean al sesiunii
29 29
 - `/api/login`
30 30
 - `/api/logout`
31
+- `POST /api/collect/dhcp-leases` — ingest DHCP lease push, protejat cu `HOST_MANAGER_DHCP_PUSH_TOKEN`
31 32
 
32 33
 Healthcheck-ul `/healthz` este disponibil doar pe backend-ul local (`127.0.0.1:8088`). Vhost-ul nginx nu îl expune.
33 34
 
+19 -0
.doc/local-hosts.md
@@ -89,6 +89,25 @@ Pe jumper, LAN-ul trebuie servit direct de `dnscrypt-proxy` pe `192.168.2.100:53
89 89
 
90 90
 `dnscrypt-proxy` folosește blocklist; `google.com` și `www.google.com` sunt allowlist-uite explicit ca să nu fie întoarse ca `0.0.0.0`.
91 91
 
92
+## Colector DHCP
93
+
94
+Routerul DHCP `192.168.2.1` împinge evenimente de lease către Madagascar Local Authority prin:
95
+
96
+```text
97
+POST https://hosts.madagascar.xdev.ro/api/collect/dhcp-leases
98
+X-DHCP-Push-Token: <HOST_MANAGER_DHCP_PUSH_TOKEN>
99
+```
100
+
101
+Endpoint-ul scrie doar în `dhcp_leases` și actualizează `data_workers.last_run_at`; nu modifică automat registry-ul de hosturi, `config/hosts.yaml` sau `config/local-hosts.tsv`.
102
+
103
+Scriptul RouterOS versionat este:
104
+
105
+```text
106
+deploy/mikrotik/dhcp-lease-push.rsc
107
+```
108
+
109
+Înainte de import, setează tokenul real în `/etc/xdev/host-manager.env` pe jumper și în variabila `hostManagerDhcpPushToken` din scriptul RouterOS. Scriptul setează `lease-script` pe serverele DHCP existente; dacă routerul are deja un `lease-script`, codul trebuie îmbinat manual ca să nu se piardă logica existentă.
110
+
92 111
 Rulează întâi dry-run:
93 112
 
94 113
 ```bash
+13 -0
deploy/jumper/README.md
@@ -37,6 +37,7 @@ sudo dnf install nginx
37 37
   scripts/host_manager.pl
38 38
   scripts/mdns_host_seed.pl
39 39
   scripts/sync_local_hosts.sh
40
+  deploy/mikrotik/dhcp-lease-push.rsc
40 41
 
41 42
 /etc/xdev/host-manager.env
42 43
 /etc/systemd/system/host-manager.service
@@ -103,3 +104,15 @@ Nu se adaugă wildcard local. Doar acest nume exact trebuie publicat.
103 104
 ## mDNS discovery
104 105
 
105 106
 `host-manager-mdns` este un listener separat care observă mDNS și scrie direct în tabelul SQLite `mdns_observations`. Listenerul nu modifică host registry-ul, `config/hosts.yaml` sau `config/local-hosts.tsv`. Sync-ul resolverului citește manifestul runtime din SQLite, nu din exportul static.
107
+
108
+## DHCP lease push
109
+
110
+Lease-urile DHCP de pe routerul `192.168.2.1` sunt colectate prin push HTTP de pe MikroTik către:
111
+
112
+```text
113
+POST https://hosts.madagascar.xdev.ro/api/collect/dhcp-leases
114
+```
115
+
116
+Setează `HOST_MANAGER_DHCP_PUSH_TOKEN` în `/etc/xdev/host-manager.env` și același token în `deploy/mikrotik/dhcp-lease-push.rsc` înainte de importul pe router. Endpoint-ul acceptă headerul `X-DHCP-Push-Token` sau `Authorization: Bearer ...` și scrie observații în `dhcp_leases`.
117
+
118
+Scriptul RouterOS folosește `lease-script`, deci trimite evenimentele noi/modificate. Dacă routerul are deja un `lease-script`, îmbină manual codul din `deploy/mikrotik/dhcp-lease-push.rsc`.
+4 -0
deploy/jumper/host-manager.env.example
@@ -12,3 +12,7 @@ HOST_MANAGER_TOTP_SECRET=CHANGE_ME_BASE32
12 12
 
13 13
 # Optional stable random value used to sign local sessions.
14 14
 HOST_MANAGER_SESSION_SECRET=CHANGE_ME_RANDOM_HEX
15
+
16
+# Shared token accepted by POST /api/collect/dhcp-leases.
17
+# This is for the MikroTik DHCP lease push hook, not for browser login.
18
+HOST_MANAGER_DHCP_PUSH_TOKEN=CHANGE_ME_DHCP_PUSH_TOKEN
+27 -0
deploy/mikrotik/dhcp-lease-push.rsc
@@ -0,0 +1,27 @@
1
+# MikroTik RouterOS DHCP lease push hook for Madagascar Local Authority.
2
+#
3
+# Edit hostManagerDhcpPushToken before import. If the DHCP server already has a
4
+# lease-script, merge this source manually instead of overwriting it.
5
+
6
+:global hostManagerDhcpPushUrl "https://hosts.madagascar.xdev.ro/api/collect/dhcp-leases";
7
+:global hostManagerDhcpPushToken "CHANGE_ME_DHCP_PUSH_TOKEN";
8
+
9
+/ip dhcp-server set [find] lease-script={
10
+    :global hostManagerDhcpPushUrl;
11
+    :global hostManagerDhcpPushToken;
12
+
13
+    :local leaseName $"lease-hostname";
14
+    :local leaseState "unbound";
15
+    :if ($leaseBound = 1) do={
16
+        :set leaseState "bound";
17
+    }
18
+
19
+    :local payload ("worker_id=dhcp-router&ip_address=" . $leaseActIP . "&mac_address=" . $leaseActMAC . "&observed_name=" . $leaseName . "&lease_state=" . $leaseState . "&bound=" . $leaseBound);
20
+    /tool fetch url=$hostManagerDhcpPushUrl \
21
+        http-method=post \
22
+        http-header-field=("Content-Type:application/x-www-form-urlencoded,X-DHCP-Push-Token:" . $hostManagerDhcpPushToken) \
23
+        http-data=$payload \
24
+        http-percent-encoding=yes \
25
+        check-certificate=no \
26
+        keep-result=no;
27
+}
+177 -0
scripts/host_manager.pl
@@ -91,6 +91,7 @@ Usage: perl scripts/host_manager.pl [--bind 127.0.0.1] [--port 8088]
91 91
 Environment:
92 92
   HOST_MANAGER_TOTP_SECRET      Base32 TOTP secret required for write access.
93 93
   HOST_MANAGER_SESSION_SECRET   Optional session signing secret.
94
+  HOST_MANAGER_DHCP_PUSH_TOKEN  Token for DHCP lease push collector.
94 95
   HOST_MANAGER_DB               Defaults to var/host-manager.sqlite.
95 96
   HOST_MANAGER_DATA             Defaults to config/hosts.yaml.
96 97
   HOST_MANAGER_LOCAL_HOSTS_TSV  Defaults to config/local-hosts.tsv.
@@ -150,6 +151,9 @@ sub handle_client {
150 151
         expire_session(\%headers);
151 152
         return send_json($client, 200, { ok => json_bool(1) }, [ "Set-Cookie: hm_session=deleted; Max-Age=0; Path=/" ]);
152 153
     }
154
+    if ($method eq 'POST' && $path eq '/api/collect/dhcp-leases') {
155
+        return collect_dhcp_leases($client, \%headers, $body);
156
+    }
153 157
 
154 158
     return send_json($client, 401, { error => 'authentication_required' }) unless is_authenticated(\%headers);
155 159
 
@@ -306,6 +310,156 @@ sub work_orders_payload {
306 310
     };
307 311
 }
308 312
 
313
+sub collect_dhcp_leases {
314
+    my ($client, $headers, $body) = @_;
315
+    my $expected = $ENV{HOST_MANAGER_DHCP_PUSH_TOKEN} || '';
316
+    return send_json($client, 503, { error => 'dhcp_push_not_configured' }) unless length $expected;
317
+
318
+    my $provided = dhcp_push_token_from_headers($headers);
319
+    return send_json($client, 401, { error => 'invalid_dhcp_push_token' }) unless token_matches($expected, $provided);
320
+
321
+    my $payload = request_payload($headers, $body);
322
+    my @leases = dhcp_payload_leases($payload);
323
+    return send_json($client, 400, { error => 'missing_dhcp_leases' }) unless @leases;
324
+
325
+    my $dbh = dbh();
326
+    my $now = iso_now();
327
+    my $worker_id = clean_id($payload->{worker_id} || $payload->{source_id} || 'dhcp-router');
328
+    $worker_id ||= 'dhcp-router';
329
+    my @stored;
330
+    with_transaction($dbh, sub {
331
+        upsert_dhcp_worker($dbh, $worker_id, $now);
332
+        for my $lease (@leases) {
333
+            my $stored = upsert_dhcp_lease($dbh, $worker_id, $lease, $now);
334
+            push @stored, $stored if $stored;
335
+        }
336
+        $dbh->do(
337
+            'UPDATE data_workers SET last_run_at = ?, updated_at = ? WHERE worker_id = ?',
338
+            undef,
339
+            $now,
340
+            $now,
341
+            $worker_id,
342
+        );
343
+    });
344
+
345
+    return send_json($client, 200, {
346
+        ok => json_bool(1),
347
+        worker_id => $worker_id,
348
+        stored => scalar(@stored),
349
+        leases => \@stored,
350
+    });
351
+}
352
+
353
+sub dhcp_push_token_from_headers {
354
+    my ($headers) = @_;
355
+    my $token = clean_scalar($headers->{'x-dhcp-push-token'} || '');
356
+    return $token if length $token;
357
+    my $authorization = clean_scalar($headers->{authorization} || '');
358
+    return $1 if $authorization =~ /\ABearer\s+(.+)\z/i;
359
+    return '';
360
+}
361
+
362
+sub token_matches {
363
+    my ($expected, $provided) = @_;
364
+    return 0 unless length($expected || '') && length($provided || '');
365
+    return 0 unless length($expected) == length($provided);
366
+    my $diff = 0;
367
+    for my $i (0 .. length($expected) - 1) {
368
+        $diff |= ord(substr($expected, $i, 1)) ^ ord(substr($provided, $i, 1));
369
+    }
370
+    return $diff == 0 ? 1 : 0;
371
+}
372
+
373
+sub dhcp_payload_leases {
374
+    my ($payload) = @_;
375
+    return () unless ref($payload) eq 'HASH';
376
+    if (ref($payload->{leases}) eq 'ARRAY') {
377
+        return grep { ref($_) eq 'HASH' } @{ $payload->{leases} };
378
+    }
379
+    return ($payload);
380
+}
381
+
382
+sub upsert_dhcp_worker {
383
+    my ($dbh, $worker_id, $now) = @_;
384
+    $dbh->do(
385
+        'INSERT INTO data_workers (worker_id, worker_type, name, status, source, last_run_at, notes, created_at, updated_at) '
386
+        . "VALUES (?, 'dhcp', 'Router DHCP leases', 'active', 'push:192.168.2.1', ?, 'DHCP lease push collector source.', ?, ?) "
387
+        . 'ON CONFLICT(worker_id) DO UPDATE SET worker_type = excluded.worker_type, name = excluded.name, status = excluded.status, '
388
+        . 'source = excluded.source, last_run_at = excluded.last_run_at, notes = excluded.notes, updated_at = excluded.updated_at',
389
+        undef,
390
+        $worker_id,
391
+        $now,
392
+        $now,
393
+        $now,
394
+    );
395
+}
396
+
397
+sub upsert_dhcp_lease {
398
+    my ($dbh, $worker_id, $lease, $now) = @_;
399
+    my $ip = clean_ip($lease->{ip_address} || $lease->{ip} || $lease->{address} || '');
400
+    my $mac = clean_mac($lease->{mac_address} || $lease->{mac} || $lease->{active_mac} || '');
401
+    return unless length $ip || length $mac;
402
+
403
+    my $name = normalize_dhcp_name($lease->{observed_name} || $lease->{host_name} || $lease->{hostname} || $lease->{name} || '');
404
+    my $state = clean_scalar($lease->{lease_state} || $lease->{state} || $lease->{status} || '');
405
+    if (!length $state && exists $lease->{bound}) {
406
+        $state = ($lease->{bound} || '') eq '1' ? 'bound' : 'unbound';
407
+    }
408
+    $state ||= 'observed';
409
+
410
+    my $lease_key = length $mac ? "$worker_id|mac|$mac" : "$worker_id|ip|$ip";
411
+    my $host_fqdn = match_dhcp_host_fqdn($dbh, $name, $ip);
412
+    my $raw = json_encode($lease);
413
+    $dbh->do(
414
+        'INSERT INTO dhcp_leases (lease_key, worker_id, host_fqdn, observed_name, ip_address, mac_address, lease_state, first_seen, last_seen, raw) '
415
+        . 'VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) '
416
+        . 'ON CONFLICT(lease_key) DO UPDATE SET host_fqdn = excluded.host_fqdn, observed_name = excluded.observed_name, '
417
+        . 'ip_address = excluded.ip_address, mac_address = excluded.mac_address, lease_state = excluded.lease_state, '
418
+        . 'last_seen = excluded.last_seen, raw = excluded.raw',
419
+        undef,
420
+        $lease_key,
421
+        $worker_id,
422
+        length($host_fqdn) ? $host_fqdn : undef,
423
+        $name,
424
+        $ip,
425
+        $mac,
426
+        $state,
427
+        $now,
428
+        $now,
429
+        $raw,
430
+    );
431
+
432
+    return {
433
+        lease_key => $lease_key,
434
+        host_fqdn => $host_fqdn,
435
+        observed_name => $name,
436
+        ip_address => $ip,
437
+        mac_address => $mac,
438
+        lease_state => $state,
439
+    };
440
+}
441
+
442
+sub match_dhcp_host_fqdn {
443
+    my ($dbh, $name, $ip) = @_;
444
+    my @names;
445
+    $name = normalize_dns_name($name || '');
446
+    if (length $name) {
447
+        push @names, $name;
448
+        push @names, "$name.madagascar.xdev.ro" unless $name =~ /\./;
449
+    }
450
+    for my $candidate (unique_preserve(@names)) {
451
+        my ($fqdn) = $dbh->selectrow_array('SELECT fqdn FROM hosts WHERE fqdn = ? AND status <> ?', undef, $candidate, 'retired');
452
+        return $fqdn if $fqdn;
453
+        ($fqdn) = $dbh->selectrow_array('SELECT host_fqdn FROM host_aliases WHERE alias_name = ? AND status = ?', undef, $candidate, 'active');
454
+        return $fqdn if $fqdn;
455
+    }
456
+    if (length($ip || '')) {
457
+        my ($fqdn) = $dbh->selectrow_array('SELECT fqdn FROM hosts WHERE (dns_ip = ? OR hosts_ip = ?) AND status <> ? ORDER BY fqdn LIMIT 1', undef, $ip, $ip, 'retired');
458
+        return $fqdn if $fqdn;
459
+    }
460
+    return '';
461
+}
462
+
309 463
 sub confirm_work_order {
310 464
     my ($client, $payload) = @_;
311 465
     my $id = clean_scalar($payload->{id} || '');
@@ -1801,6 +1955,29 @@ sub clean_certificate_id {
1801 1955
     return $value =~ /\A[A-Za-z0-9_.-]+\z/ ? $value : '';
1802 1956
 }
1803 1957
 
1958
+sub clean_ip {
1959
+    my ($value) = @_;
1960
+    $value = clean_scalar($value);
1961
+    return $value if $value =~ /\A(?:25[0-5]|2[0-4]\d|1?\d?\d)(?:\.(?:25[0-5]|2[0-4]\d|1?\d?\d)){3}\z/;
1962
+    return '';
1963
+}
1964
+
1965
+sub clean_mac {
1966
+    my ($value) = @_;
1967
+    $value = lc clean_scalar($value);
1968
+    $value =~ s/-/:/g;
1969
+    return $value if $value =~ /\A[0-9a-f]{2}(?::[0-9a-f]{2}){5}\z/;
1970
+    return '';
1971
+}
1972
+
1973
+sub normalize_dhcp_name {
1974
+    my ($value) = @_;
1975
+    $value = normalize_dns_name($value || '');
1976
+    $value =~ s/[^a-z0-9_.-]+/-/g;
1977
+    $value =~ s/^-+|-+$//g;
1978
+    return $value;
1979
+}
1980
+
1804 1981
 sub clean_scalar {
1805 1982
     my ($value) = @_;
1806 1983
     $value = '' unless defined $value;