Implement phase 1 foundations
All checks were successful
build-image / docker (push) Successful in 53s
All checks were successful
build-image / docker (push) Successful in 53s
This commit is contained in:
commit
6f7549d439
5
.dockerignore
Normal file
5
.dockerignore
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
.git
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
data
|
||||||
|
.pi-subagents
|
||||||
15
.env.example
Normal file
15
.env.example
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
LOG_GUARDIAN_ADDR=:8080
|
||||||
|
LOG_GUARDIAN_DATA_PATH=./data/config.json
|
||||||
|
LOG_GUARDIAN_POLL_INTERVAL=1m
|
||||||
|
LOG_GUARDIAN_DRY_RUN_ACTIONS=true
|
||||||
|
LOG_GUARDIAN_AUTH_USERNAME=admin
|
||||||
|
LOG_GUARDIAN_AUTH_PASSWORD=change-me
|
||||||
|
LOKI_URL=http://localhost:3100
|
||||||
|
LOKI_TENANT_ID=
|
||||||
|
LOKI_USERNAME=
|
||||||
|
LOKI_PASSWORD=
|
||||||
|
LOG_GUARDIAN_ANALYSIS_PROVIDER=none
|
||||||
|
LOG_GUARDIAN_ANALYSIS_ENDPOINT=
|
||||||
|
LOG_GUARDIAN_ANALYSIS_TOKEN=
|
||||||
|
LOG_GUARDIAN_OLLAMA_URL=http://ollama:11434
|
||||||
|
LOG_GUARDIAN_OLLAMA_MODEL=llama3.1
|
||||||
13
.gitea/workflows/build-image.yml
Normal file
13
.gitea/workflows/build-image.yml
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
name: build-image
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
docker:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- name: Build image
|
||||||
|
run: make docker-build IMAGE_TAG=${{ gitea.sha }}
|
||||||
17
.github/workflows/ci.yml
vendored
Normal file
17
.github/workflows/ci.yml
vendored
Normal file
@ -0,0 +1,17 @@
|
|||||||
|
name: ci
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
pull_request:
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
test:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-go@v5
|
||||||
|
with:
|
||||||
|
go-version: '1.23'
|
||||||
|
- run: make validate-structure
|
||||||
|
- run: make go-fmt
|
||||||
|
- run: make go-test
|
||||||
9
.gitignore
vendored
Normal file
9
.gitignore
vendored
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
data/
|
||||||
|
dist/
|
||||||
|
build/
|
||||||
|
coverage/
|
||||||
|
*.log
|
||||||
|
.DS_Store
|
||||||
33
AGENTS.md
Normal file
33
AGENTS.md
Normal file
@ -0,0 +1,33 @@
|
|||||||
|
# AGENTS.md
|
||||||
|
|
||||||
|
This repository is for a private log monitoring, alerting, and controlled remediation service built around Loki.
|
||||||
|
|
||||||
|
## Hard rules
|
||||||
|
|
||||||
|
- Server-side code must be Go.
|
||||||
|
- The web UI is served by Go; do not add a server-side JavaScript runtime.
|
||||||
|
- Never log alert destination secrets, API keys, tokens, AI prompts containing private logs, or full raw log payloads unless explicitly configured for a local-only debug session.
|
||||||
|
- Do not add external cloud/AI/data integrations without explicit user approval.
|
||||||
|
- Automatic remediation must be opt-in per rule and should default to dry-run.
|
||||||
|
- AI analysis is advisory by default; AI-generated actions must not execute unless a rule explicitly allows that action and passes guardrails.
|
||||||
|
- Preserve local data ownership: runtime configuration is stored in mounted local files unless a later phase explicitly changes storage.
|
||||||
|
|
||||||
|
## Required artifacts per phase
|
||||||
|
|
||||||
|
- `project-docs/status/phase-N-summary.md`
|
||||||
|
- `project-docs/status/phase-N-test-results.md`
|
||||||
|
- `project-docs/status/phase-N-open-issues.md`
|
||||||
|
|
||||||
|
## Required validation before advancing
|
||||||
|
|
||||||
|
Run the checks required by the phase, including as applicable:
|
||||||
|
|
||||||
|
- formatting/lint
|
||||||
|
- unit tests
|
||||||
|
- Docker build
|
||||||
|
- Loki query validation
|
||||||
|
- alert delivery validation
|
||||||
|
- remediation dry-run evidence
|
||||||
|
- security/privacy review for logs, prompts, secrets, and action permissions
|
||||||
|
|
||||||
|
If any required check fails, the phase is not complete.
|
||||||
31
Makefile
Normal file
31
Makefile
Normal file
@ -0,0 +1,31 @@
|
|||||||
|
IMAGE_TAG ?= local
|
||||||
|
REGISTRY ?= gitea.wayfinderak.com
|
||||||
|
IMAGE_NAMESPACE ?= wayfinderak
|
||||||
|
IMAGE_PREFIX ?= log-guardian
|
||||||
|
IMAGE_REF ?= $(REGISTRY)/$(IMAGE_NAMESPACE)/$(IMAGE_PREFIX):$(IMAGE_TAG)
|
||||||
|
DOCKER_BUILD_FLAGS ?=
|
||||||
|
|
||||||
|
.PHONY: validate-structure go-fmt go-test docker-build ci-fast
|
||||||
|
|
||||||
|
validate-structure:
|
||||||
|
@test -f AGENTS.md
|
||||||
|
@test -f README.md
|
||||||
|
@test -f project-docs/status/phase-0-summary.md
|
||||||
|
@test -f project-docs/status/phase-0-test-results.md
|
||||||
|
@test -f project-docs/status/phase-0-open-issues.md
|
||||||
|
@test -f project-docs/status/phase-1-summary.md
|
||||||
|
@test -f project-docs/status/phase-1-test-results.md
|
||||||
|
@test -f project-docs/status/phase-1-open-issues.md
|
||||||
|
@test -f deploy/loki/docker-compose.yml
|
||||||
|
@echo "Structure validation passed"
|
||||||
|
|
||||||
|
go-fmt:
|
||||||
|
@test -z "$$(gofmt -l cmd internal)"
|
||||||
|
|
||||||
|
go-test:
|
||||||
|
go test ./...
|
||||||
|
|
||||||
|
docker-build:
|
||||||
|
docker build $(DOCKER_BUILD_FLAGS) -f docker/server.Dockerfile -t $(IMAGE_REF) .
|
||||||
|
|
||||||
|
ci-fast: validate-structure go-fmt go-test docker-build
|
||||||
92
README.md
Normal file
92
README.md
Normal file
@ -0,0 +1,92 @@
|
|||||||
|
# Log Guardian
|
||||||
|
|
||||||
|
Private Loki-backed service for log monitoring, alerting, and guarded remediation.
|
||||||
|
|
||||||
|
Current implementation includes the Phase 1 foundations:
|
||||||
|
|
||||||
|
- watches configured LogQL rules;
|
||||||
|
- records incidents when thresholds are met;
|
||||||
|
- enforces per-rule cooldowns;
|
||||||
|
- protects the UI with basic auth while leaving `/healthz` open;
|
||||||
|
- provides Loki label/query diagnostics;
|
||||||
|
- supports UI-configurable alert channels with real ntfy delivery;
|
||||||
|
- keeps remediation in dry-run by default;
|
||||||
|
- includes local-only Ollama analysis support, disabled by default.
|
||||||
|
|
||||||
|
## Run locally
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cp .env.example .env
|
||||||
|
# edit env values, then export them or use your shell dotenv helper
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
Open <http://localhost:8080>.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
| Variable | Default | Purpose |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `LOG_GUARDIAN_ADDR` | `:8080` | HTTP listen address. |
|
||||||
|
| `LOG_GUARDIAN_DATA_PATH` | `/data/config.json` | Local JSON config. |
|
||||||
|
| `LOG_GUARDIAN_POLL_INTERVAL` | `1m` | Rule evaluation interval. |
|
||||||
|
| `LOG_GUARDIAN_DRY_RUN_ACTIONS` | `true` | Prevents real remediation actions. |
|
||||||
|
| `LOG_GUARDIAN_AUTH_USERNAME` / `LOG_GUARDIAN_AUTH_PASSWORD` | empty | Required basic auth credentials for UI/API routes except `/healthz`. |
|
||||||
|
| `LOKI_URL` | `http://loki:3100` | Loki base URL. |
|
||||||
|
| `LOKI_TENANT_ID` | empty | Optional Loki tenant header. |
|
||||||
|
| `LOKI_USERNAME` / `LOKI_PASSWORD` | empty | Optional basic auth. |
|
||||||
|
| `LOG_GUARDIAN_ANALYSIS_PROVIDER` | `none` | `none` or local-only `ollama`. |
|
||||||
|
| `LOG_GUARDIAN_OLLAMA_URL` / `LOG_GUARDIAN_OLLAMA_MODEL` | `http://ollama:11434` / `llama3.1` | Local Ollama analysis config. |
|
||||||
|
|
||||||
|
## Initial Loki setup
|
||||||
|
|
||||||
|
A starter Loki + Promtail stack is in `deploy/loki/`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd deploy/loki
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Promtail is configured to discover Docker containers and attach useful labels:
|
||||||
|
|
||||||
|
- `container`
|
||||||
|
- `service`
|
||||||
|
- `stack`
|
||||||
|
- `stream`
|
||||||
|
|
||||||
|
Example LogQL rules:
|
||||||
|
|
||||||
|
```logql
|
||||||
|
{service="api"} |= "error"
|
||||||
|
{stack="second-brain"} |~ "(?i)(panic|fatal|exception)"
|
||||||
|
{container=~".*postgres.*"} |= "database system is ready"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deploy in Portainer
|
||||||
|
|
||||||
|
Use `deploy/portainer-stack.yml` with values from `deploy/portainer.env.example`.
|
||||||
|
|
||||||
|
Keep this set while testing:
|
||||||
|
|
||||||
|
```text
|
||||||
|
LOG_GUARDIAN_DRY_RUN_ACTIONS=true
|
||||||
|
LOG_GUARDIAN_ANALYSIS_PROVIDER=none
|
||||||
|
```
|
||||||
|
|
||||||
|
## Build and validate
|
||||||
|
|
||||||
|
```sh
|
||||||
|
make ci-fast
|
||||||
|
```
|
||||||
|
|
||||||
|
## Safety model
|
||||||
|
|
||||||
|
Automatic remediation is intentionally constrained:
|
||||||
|
|
||||||
|
1. A rule must match.
|
||||||
|
2. The configured threshold must be exceeded.
|
||||||
|
3. The action must be explicitly enabled on that rule.
|
||||||
|
4. Global dry-run must be disabled.
|
||||||
|
5. Future AI recommendations are advisory unless a rule explicitly allows a matching action type.
|
||||||
|
|
||||||
|
Do not enable production remediation until a later phase explicitly approves concrete action types and guardrails.
|
||||||
49
cmd/server/main.go
Normal file
49
cmd/server/main.go
Normal file
@ -0,0 +1,49 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/actions"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/config"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/rules"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/server"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
cfg := config.Load()
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
|
||||||
|
fileStore := store.NewFileStore(cfg.DataPath)
|
||||||
|
lokiClient := loki.New(cfg.LokiURL, cfg.LokiTenantID, cfg.LokiUsername, cfg.LokiPassword)
|
||||||
|
var analyzer analysis.Analyzer = analysis.NoopAnalyzer{}
|
||||||
|
if cfg.AnalysisProvider == "ollama" {
|
||||||
|
analyzer = analysis.NewOllamaAnalyzer(cfg.OllamaURL, cfg.OllamaModel)
|
||||||
|
}
|
||||||
|
actionRunner := actions.NewRunner(cfg.DryRunActions)
|
||||||
|
dispatcher := alerts.NewDispatcher()
|
||||||
|
engine := rules.NewEngine(fileStore, lokiClient, analyzer, actionRunner, dispatcher)
|
||||||
|
go engine.Start(ctx, cfg.PollInterval)
|
||||||
|
|
||||||
|
httpServer := &http.Server{Addr: cfg.Addr, Handler: server.New(fileStore, engine, lokiClient, dispatcher, cfg.AuthUsername, cfg.AuthPassword).Routes()}
|
||||||
|
go func() {
|
||||||
|
<-ctx.Done()
|
||||||
|
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = httpServer.Shutdown(shutdownCtx)
|
||||||
|
}()
|
||||||
|
|
||||||
|
log.Printf("log-guardian listening on %s", cfg.Addr)
|
||||||
|
if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
24
deploy/loki/docker-compose.yml
Normal file
24
deploy/loki/docker-compose.yml
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
services:
|
||||||
|
loki:
|
||||||
|
image: grafana/loki:3.2.1
|
||||||
|
command: -config.file=/etc/loki/local-config.yaml
|
||||||
|
ports:
|
||||||
|
- "${LOKI_PORT:-3100}:3100"
|
||||||
|
volumes:
|
||||||
|
- loki_data:/loki
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
promtail:
|
||||||
|
image: grafana/promtail:3.2.1
|
||||||
|
command: -config.file=/etc/promtail/config.yml
|
||||||
|
volumes:
|
||||||
|
- /var/log:/var/log:ro
|
||||||
|
- /var/lib/docker/containers:/var/lib/docker/containers:ro
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
- ./promtail-config.yml:/etc/promtail/config.yml:ro
|
||||||
|
depends_on:
|
||||||
|
- loki
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
loki_data:
|
||||||
25
deploy/loki/promtail-config.yml
Normal file
25
deploy/loki/promtail-config.yml
Normal file
@ -0,0 +1,25 @@
|
|||||||
|
server:
|
||||||
|
http_listen_port: 9080
|
||||||
|
grpc_listen_port: 0
|
||||||
|
|
||||||
|
positions:
|
||||||
|
filename: /tmp/positions.yaml
|
||||||
|
|
||||||
|
clients:
|
||||||
|
- url: http://loki:3100/loki/api/v1/push
|
||||||
|
|
||||||
|
scrape_configs:
|
||||||
|
- job_name: docker
|
||||||
|
docker_sd_configs:
|
||||||
|
- host: unix:///var/run/docker.sock
|
||||||
|
refresh_interval: 10s
|
||||||
|
relabel_configs:
|
||||||
|
- source_labels: ['__meta_docker_container_name']
|
||||||
|
regex: '/(.*)'
|
||||||
|
target_label: 'container'
|
||||||
|
- source_labels: ['__meta_docker_container_label_com_docker_compose_service']
|
||||||
|
target_label: 'service'
|
||||||
|
- source_labels: ['__meta_docker_container_label_com_docker_compose_project']
|
||||||
|
target_label: 'stack'
|
||||||
|
- source_labels: ['__meta_docker_container_log_stream']
|
||||||
|
target_label: 'stream'
|
||||||
28
deploy/portainer-stack.yml
Normal file
28
deploy/portainer-stack.yml
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
services:
|
||||||
|
log-guardian:
|
||||||
|
image: gitea.wayfinderak.com/wayfinderak/log-guardian:${IMAGE_TAG:-latest}
|
||||||
|
environment:
|
||||||
|
LOG_GUARDIAN_ADDR: :8080
|
||||||
|
LOG_GUARDIAN_DATA_PATH: /data/config.json
|
||||||
|
LOG_GUARDIAN_POLL_INTERVAL: ${LOG_GUARDIAN_POLL_INTERVAL:-1m}
|
||||||
|
LOG_GUARDIAN_DRY_RUN_ACTIONS: ${LOG_GUARDIAN_DRY_RUN_ACTIONS:-true}
|
||||||
|
LOKI_URL: ${LOKI_URL:-http://loki:3100}
|
||||||
|
LOKI_TENANT_ID: ${LOKI_TENANT_ID:-}
|
||||||
|
LOKI_USERNAME: ${LOKI_USERNAME:-}
|
||||||
|
LOKI_PASSWORD: ${LOKI_PASSWORD:-}
|
||||||
|
LOG_GUARDIAN_ANALYSIS_PROVIDER: ${LOG_GUARDIAN_ANALYSIS_PROVIDER:-none}
|
||||||
|
LOG_GUARDIAN_ANALYSIS_ENDPOINT: ${LOG_GUARDIAN_ANALYSIS_ENDPOINT:-}
|
||||||
|
LOG_GUARDIAN_ANALYSIS_TOKEN: ${LOG_GUARDIAN_ANALYSIS_TOKEN:-}
|
||||||
|
volumes:
|
||||||
|
- log_guardian_data:/data
|
||||||
|
ports:
|
||||||
|
- "${LOG_GUARDIAN_PORT:-8081}:8080"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8080/healthz"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 5
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
log_guardian_data:
|
||||||
11
deploy/portainer.env.example
Normal file
11
deploy/portainer.env.example
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
IMAGE_TAG=latest
|
||||||
|
LOG_GUARDIAN_PORT=8081
|
||||||
|
LOG_GUARDIAN_POLL_INTERVAL=1m
|
||||||
|
LOG_GUARDIAN_DRY_RUN_ACTIONS=true
|
||||||
|
LOKI_URL=http://loki:3100
|
||||||
|
LOKI_TENANT_ID=
|
||||||
|
LOKI_USERNAME=
|
||||||
|
LOKI_PASSWORD=
|
||||||
|
LOG_GUARDIAN_ANALYSIS_PROVIDER=none
|
||||||
|
LOG_GUARDIAN_ANALYSIS_ENDPOINT=
|
||||||
|
LOG_GUARDIAN_ANALYSIS_TOKEN=
|
||||||
19
docker/server.Dockerfile
Normal file
19
docker/server.Dockerfile
Normal file
@ -0,0 +1,19 @@
|
|||||||
|
FROM golang:1.23-alpine AS build
|
||||||
|
WORKDIR /src
|
||||||
|
COPY go.mod ./
|
||||||
|
COPY cmd ./cmd
|
||||||
|
COPY internal ./internal
|
||||||
|
COPY web ./web
|
||||||
|
RUN go build -trimpath -ldflags="-s -w" -o /out/log-guardian ./cmd/server
|
||||||
|
|
||||||
|
FROM alpine:3.20
|
||||||
|
LABEL org.opencontainers.image.title="log-guardian"
|
||||||
|
LABEL org.opencontainers.image.description="Loki-backed log monitoring, alerting, and guarded remediation service."
|
||||||
|
WORKDIR /app
|
||||||
|
RUN adduser -D -H guardian && mkdir -p /data && chown guardian:guardian /data
|
||||||
|
COPY --from=build /out/log-guardian /usr/local/bin/log-guardian
|
||||||
|
COPY --from=build /src/web ./web
|
||||||
|
USER guardian
|
||||||
|
EXPOSE 8080
|
||||||
|
VOLUME ["/data"]
|
||||||
|
CMD ["/usr/local/bin/log-guardian"]
|
||||||
3
go.mod
Normal file
3
go.mod
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
module gitea.wayfinderak.com/wayfinderak/log-guardian
|
||||||
|
|
||||||
|
go 1.23
|
||||||
65
internal/actions/actions.go
Normal file
65
internal/actions/actions.go
Normal file
@ -0,0 +1,65 @@
|
|||||||
|
package actions
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Result struct {
|
||||||
|
Action string `json:"action"`
|
||||||
|
DryRun bool `json:"dry_run"`
|
||||||
|
Detail string `json:"detail"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Runner struct {
|
||||||
|
dryRun bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewRunner(dryRun bool) *Runner { return &Runner{dryRun: dryRun} }
|
||||||
|
|
||||||
|
func (r *Runner) Run(ctx context.Context, rule store.Rule, action store.Action) (Result, error) {
|
||||||
|
_ = ctx
|
||||||
|
if !action.Enabled {
|
||||||
|
return Result{Action: action.Type, DryRun: r.effectiveDryRun(action), Detail: "disabled"}, nil
|
||||||
|
}
|
||||||
|
switch action.Type {
|
||||||
|
case "generic_webhook", "webhook":
|
||||||
|
url := action.Params["url"]
|
||||||
|
if url == "" {
|
||||||
|
return Result{}, fmt.Errorf("webhook action requires url")
|
||||||
|
}
|
||||||
|
return Result{Action: action.Type, DryRun: true, Detail: "would POST webhook for rule " + rule.Name}, nil
|
||||||
|
case "record_recommendation":
|
||||||
|
return Result{Action: action.Type, DryRun: false, Detail: "recommendation recorded for operator review"}, nil
|
||||||
|
case "allowlisted_command", "command":
|
||||||
|
id := action.Params["command_id"]
|
||||||
|
if id == "" {
|
||||||
|
id = action.Params["id"]
|
||||||
|
}
|
||||||
|
if id == "" {
|
||||||
|
return Result{}, fmt.Errorf("allowlisted command requires command_id")
|
||||||
|
}
|
||||||
|
return Result{Action: action.Type, DryRun: true, Detail: "would run allowlisted command " + id}, nil
|
||||||
|
case "portainer_restart":
|
||||||
|
service := action.Params["service"]
|
||||||
|
if service == "" {
|
||||||
|
return Result{}, fmt.Errorf("portainer_restart requires service")
|
||||||
|
}
|
||||||
|
return Result{Action: action.Type, DryRun: true, Detail: "would restart Portainer service " + service}, nil
|
||||||
|
default:
|
||||||
|
return Result{}, fmt.Errorf("unsupported action type %q", action.Type)
|
||||||
|
}
|
||||||
|
return Result{}, fmt.Errorf("unsupported action type %q", action.Type)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) effectiveDryRun(action store.Action) bool {
|
||||||
|
if r.dryRun {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if action.DryRun == nil {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return *action.DryRun
|
||||||
|
}
|
||||||
108
internal/alerts/alerts.go
Normal file
108
internal/alerts/alerts.go
Normal file
@ -0,0 +1,108 @@
|
|||||||
|
package alerts
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Result struct {
|
||||||
|
ChannelID string
|
||||||
|
Detail string
|
||||||
|
}
|
||||||
|
|
||||||
|
type Dispatcher struct {
|
||||||
|
http *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewDispatcher() *Dispatcher { return &Dispatcher{http: &http.Client{Timeout: 15 * time.Second}} }
|
||||||
|
|
||||||
|
func (d *Dispatcher) Send(ctx context.Context, rule store.Rule, finding analysis.Finding, channels []store.AlertChannel) ([]Result, error) {
|
||||||
|
if len(channels) == 0 {
|
||||||
|
log.Printf("alert for rule %q: %s", rule.Name, finding.Summary)
|
||||||
|
return []Result{{Detail: "no channels configured; logged alert intent"}}, nil
|
||||||
|
}
|
||||||
|
var results []Result
|
||||||
|
var firstErr error
|
||||||
|
for _, channel := range channels {
|
||||||
|
if !channel.Enabled {
|
||||||
|
results = append(results, Result{ChannelID: channel.ID, Detail: "disabled"})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var err error
|
||||||
|
switch channel.Type {
|
||||||
|
case "ntfy":
|
||||||
|
err = d.sendNtfy(ctx, rule, finding, channel)
|
||||||
|
case "gotify", "generic_webhook", "smtp":
|
||||||
|
err = fmt.Errorf("%s delivery is scaffolded but not implemented in phase 1", channel.Type)
|
||||||
|
default:
|
||||||
|
err = fmt.Errorf("unsupported channel type %q", channel.Type)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
if firstErr == nil {
|
||||||
|
firstErr = err
|
||||||
|
}
|
||||||
|
results = append(results, Result{ChannelID: channel.ID, Detail: err.Error()})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
results = append(results, Result{ChannelID: channel.ID, Detail: "sent"})
|
||||||
|
}
|
||||||
|
return results, firstErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *Dispatcher) Test(ctx context.Context, channel store.AlertChannel) error {
|
||||||
|
finding := analysis.Finding{Summary: "Log Guardian test notification", Confidence: "test"}
|
||||||
|
rule := store.Rule{Name: "Test alert", Severity: "info"}
|
||||||
|
switch channel.Type {
|
||||||
|
case "ntfy":
|
||||||
|
return d.sendNtfy(ctx, rule, finding, channel)
|
||||||
|
case "gotify", "generic_webhook", "smtp":
|
||||||
|
return fmt.Errorf("%s delivery is scaffolded but not implemented in phase 1", channel.Type)
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("unsupported channel type %q", channel.Type)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *Dispatcher) sendNtfy(ctx context.Context, rule store.Rule, finding analysis.Finding, channel store.AlertChannel) error {
|
||||||
|
server := strings.TrimRight(channel.Params["server_url"], "/")
|
||||||
|
topic := strings.Trim(channel.Params["topic"], "/")
|
||||||
|
if server == "" || topic == "" {
|
||||||
|
return fmt.Errorf("ntfy requires server_url and topic")
|
||||||
|
}
|
||||||
|
target, err := url.JoinPath(server, topic)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
body := rule.Name + ": " + finding.Summary
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, target, bytes.NewBufferString(body))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
req.Header.Set("Title", "Log Guardian: "+rule.Severity)
|
||||||
|
if priority := channel.Params["priority"]; priority != "" {
|
||||||
|
req.Header.Set("Priority", priority)
|
||||||
|
}
|
||||||
|
if tags := channel.Params["tags"]; tags != "" {
|
||||||
|
req.Header.Set("Tags", tags)
|
||||||
|
}
|
||||||
|
if token := channel.Params["token"]; token != "" {
|
||||||
|
req.Header.Set("Authorization", "Bearer "+token)
|
||||||
|
}
|
||||||
|
resp, err := d.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode < 200 || resp.StatusCode > 299 {
|
||||||
|
return fmt.Errorf("ntfy delivery failed: status %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
31
internal/alerts/alerts_test.go
Normal file
31
internal/alerts/alerts_test.go
Normal file
@ -0,0 +1,31 @@
|
|||||||
|
package alerts
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestNtfyDelivery(t *testing.T) {
|
||||||
|
var gotAuth string
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path != "/alerts" {
|
||||||
|
t.Fatalf("unexpected path %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
gotAuth = r.Header.Get("Authorization")
|
||||||
|
w.WriteHeader(http.StatusNoContent)
|
||||||
|
}))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
d := NewDispatcher()
|
||||||
|
_, err := d.Send(t.Context(), store.Rule{Name: "API", Severity: "warning"}, analysis.Finding{Summary: "failed"}, []store.AlertChannel{{ID: "c1", Name: "ntfy", Type: "ntfy", Enabled: true, Params: map[string]string{"server_url": ts.URL, "topic": "alerts", "token": "secret"}}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if gotAuth != "Bearer secret" {
|
||||||
|
t.Fatalf("missing auth header: %q", gotAuth)
|
||||||
|
}
|
||||||
|
}
|
||||||
160
internal/analysis/analysis.go
Normal file
160
internal/analysis/analysis.go
Normal file
@ -0,0 +1,160 @@
|
|||||||
|
package analysis
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Finding struct {
|
||||||
|
Summary string `json:"summary"`
|
||||||
|
RecommendedActions []string `json:"recommended_actions,omitempty"`
|
||||||
|
Confidence string `json:"confidence"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Analyzer interface {
|
||||||
|
Analyze(ctx context.Context, rule store.Rule, matches []loki.Match) (Finding, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type NoopAnalyzer struct{}
|
||||||
|
|
||||||
|
func (NoopAnalyzer) Analyze(ctx context.Context, rule store.Rule, matches []loki.Match) (Finding, error) {
|
||||||
|
_ = ctx
|
||||||
|
return Finding{Summary: summarize(rule, matches), Confidence: "rule-only"}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type OllamaAnalyzer struct {
|
||||||
|
URL string
|
||||||
|
Model string
|
||||||
|
http *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewOllamaAnalyzer(url, model string) *OllamaAnalyzer {
|
||||||
|
return &OllamaAnalyzer{URL: strings.TrimRight(url, "/"), Model: model, http: &http.Client{Timeout: 60 * time.Second}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *OllamaAnalyzer) Analyze(ctx context.Context, rule store.Rule, matches []loki.Match) (Finding, error) {
|
||||||
|
if a.URL == "" || a.Model == "" {
|
||||||
|
return NoopAnalyzer{}.Analyze(ctx, rule, matches)
|
||||||
|
}
|
||||||
|
prompt := buildPrompt(rule, matches)
|
||||||
|
body, _ := json.Marshal(map[string]any{"model": a.Model, "prompt": prompt, "stream": false})
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.URL+"/api/generate", bytes.NewReader(body))
|
||||||
|
if err != nil {
|
||||||
|
return Finding{}, err
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
resp, err := a.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return Finding{}, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode < 200 || resp.StatusCode > 299 {
|
||||||
|
return Finding{}, fmt.Errorf("ollama analysis failed: status %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
var raw struct {
|
||||||
|
Response string `json:"response"`
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&raw); err != nil {
|
||||||
|
return Finding{}, err
|
||||||
|
}
|
||||||
|
summary := strings.TrimSpace(raw.Response)
|
||||||
|
if summary == "" {
|
||||||
|
return NoopAnalyzer{}.Analyze(ctx, rule, matches)
|
||||||
|
}
|
||||||
|
if len(summary) > 500 {
|
||||||
|
summary = summary[:500] + "..."
|
||||||
|
}
|
||||||
|
return Finding{Summary: summary, Confidence: "ollama-local"}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func buildPrompt(rule store.Rule, matches []loki.Match) string {
|
||||||
|
var b strings.Builder
|
||||||
|
b.WriteString("You are analyzing private application logs locally. Summarize likely cause and operator next steps in 4 concise sentences or fewer. Do not suggest destructive actions.\n")
|
||||||
|
b.WriteString("Rule: ")
|
||||||
|
b.WriteString(rule.Name)
|
||||||
|
b.WriteString("\nSeverity: ")
|
||||||
|
b.WriteString(rule.Severity)
|
||||||
|
b.WriteString("\nSanitized samples:\n")
|
||||||
|
limit := len(matches)
|
||||||
|
if limit > 5 {
|
||||||
|
limit = 5
|
||||||
|
}
|
||||||
|
for i := 0; i < limit; i++ {
|
||||||
|
line := sanitizeLine(matches[i].Line)
|
||||||
|
line = redactSecrets(line)
|
||||||
|
if len(line) > 300 {
|
||||||
|
line = line[:300] + "..."
|
||||||
|
}
|
||||||
|
b.WriteString("- ")
|
||||||
|
b.WriteString(line)
|
||||||
|
b.WriteByte('\n')
|
||||||
|
}
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
func summarize(rule store.Rule, matches []loki.Match) string {
|
||||||
|
if len(matches) == 0 {
|
||||||
|
return "No matching log entries."
|
||||||
|
}
|
||||||
|
line := redactSecrets(sanitizeLine(matches[0].Line))
|
||||||
|
if len(line) > 180 {
|
||||||
|
line = line[:180] + "..."
|
||||||
|
}
|
||||||
|
return rule.Name + " matched " + plural(len(matches), "log entry", "log entries") + "; first match: " + line
|
||||||
|
}
|
||||||
|
|
||||||
|
func sanitizeLine(line string) string {
|
||||||
|
line = strings.ReplaceAll(line, "\n", " ")
|
||||||
|
line = strings.ReplaceAll(line, "\r", " ")
|
||||||
|
return strings.TrimSpace(line)
|
||||||
|
}
|
||||||
|
|
||||||
|
func redactSecrets(line string) string {
|
||||||
|
fields := strings.Fields(line)
|
||||||
|
for i, field := range fields {
|
||||||
|
lower := strings.ToLower(field)
|
||||||
|
if strings.Contains(lower, "token=") || strings.Contains(lower, "password=") || strings.Contains(lower, "secret=") || strings.Contains(lower, "apikey=") || strings.Contains(lower, "api_key=") || strings.HasPrefix(lower, "bearer") {
|
||||||
|
fields[i] = redactField(field)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return strings.Join(fields, " ")
|
||||||
|
}
|
||||||
|
|
||||||
|
func redactField(field string) string {
|
||||||
|
idx := strings.IndexAny(field, "=:")
|
||||||
|
if idx < 0 {
|
||||||
|
return "[REDACTED]"
|
||||||
|
}
|
||||||
|
prefix := strings.TrimRightFunc(field[:idx], func(r rune) bool { return unicode.IsSpace(r) })
|
||||||
|
return prefix + "=[REDACTED]"
|
||||||
|
}
|
||||||
|
|
||||||
|
func plural(count int, singular, plural string) string {
|
||||||
|
if count == 1 {
|
||||||
|
return "1 " + singular
|
||||||
|
}
|
||||||
|
return strconvItoa(count) + " " + plural
|
||||||
|
}
|
||||||
|
|
||||||
|
func strconvItoa(i int) string {
|
||||||
|
if i == 0 {
|
||||||
|
return "0"
|
||||||
|
}
|
||||||
|
var b [20]byte
|
||||||
|
bp := len(b)
|
||||||
|
for i > 0 {
|
||||||
|
bp--
|
||||||
|
b[bp] = byte('0' + i%10)
|
||||||
|
i /= 10
|
||||||
|
}
|
||||||
|
return string(b[bp:])
|
||||||
|
}
|
||||||
10
internal/analysis/analysis_test.go
Normal file
10
internal/analysis/analysis_test.go
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
package analysis
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestSanitizeLine(t *testing.T) {
|
||||||
|
got := sanitizeLine(" error\nwith\rspaces ")
|
||||||
|
if got != "error with spaces" {
|
||||||
|
t.Fatalf("got %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
83
internal/config/config.go
Normal file
83
internal/config/config.go
Normal file
@ -0,0 +1,83 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Config struct {
|
||||||
|
Addr string
|
||||||
|
DataPath string
|
||||||
|
PollInterval time.Duration
|
||||||
|
LokiURL string
|
||||||
|
LokiTenantID string
|
||||||
|
LokiUsername string
|
||||||
|
LokiPassword string
|
||||||
|
DryRunActions bool
|
||||||
|
AnalysisProvider string
|
||||||
|
AnalysisEndpoint string
|
||||||
|
AnalysisToken string
|
||||||
|
OllamaURL string
|
||||||
|
OllamaModel string
|
||||||
|
AuthUsername string
|
||||||
|
AuthPassword string
|
||||||
|
}
|
||||||
|
|
||||||
|
func Load() Config {
|
||||||
|
return Config{
|
||||||
|
Addr: env("LOG_GUARDIAN_ADDR", ":8080"),
|
||||||
|
DataPath: env("LOG_GUARDIAN_DATA_PATH", "/data/config.json"),
|
||||||
|
PollInterval: durationEnv("LOG_GUARDIAN_POLL_INTERVAL", 1*time.Minute),
|
||||||
|
LokiURL: trimRightSlash(env("LOKI_URL", "http://loki:3100")),
|
||||||
|
LokiTenantID: env("LOKI_TENANT_ID", ""),
|
||||||
|
LokiUsername: env("LOKI_USERNAME", ""),
|
||||||
|
LokiPassword: env("LOKI_PASSWORD", ""),
|
||||||
|
DryRunActions: boolEnv("LOG_GUARDIAN_DRY_RUN_ACTIONS", true),
|
||||||
|
AnalysisProvider: env("LOG_GUARDIAN_ANALYSIS_PROVIDER", "none"),
|
||||||
|
AnalysisEndpoint: env("LOG_GUARDIAN_ANALYSIS_ENDPOINT", ""),
|
||||||
|
AnalysisToken: env("LOG_GUARDIAN_ANALYSIS_TOKEN", ""),
|
||||||
|
OllamaURL: trimRightSlash(env("LOG_GUARDIAN_OLLAMA_URL", "http://ollama:11434")),
|
||||||
|
OllamaModel: env("LOG_GUARDIAN_OLLAMA_MODEL", "llama3.1"),
|
||||||
|
AuthUsername: env("LOG_GUARDIAN_AUTH_USERNAME", ""),
|
||||||
|
AuthPassword: env("LOG_GUARDIAN_AUTH_PASSWORD", ""),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func env(key, fallback string) string {
|
||||||
|
if value := os.Getenv(key); value != "" {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
func boolEnv(key string, fallback bool) bool {
|
||||||
|
value := os.Getenv(key)
|
||||||
|
if value == "" {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
parsed, err := strconv.ParseBool(value)
|
||||||
|
if err != nil {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
return parsed
|
||||||
|
}
|
||||||
|
|
||||||
|
func durationEnv(key string, fallback time.Duration) time.Duration {
|
||||||
|
value := os.Getenv(key)
|
||||||
|
if value == "" {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
parsed, err := time.ParseDuration(value)
|
||||||
|
if err != nil {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
return parsed
|
||||||
|
}
|
||||||
|
|
||||||
|
func trimRightSlash(value string) string {
|
||||||
|
for len(value) > 1 && value[len(value)-1] == '/' {
|
||||||
|
value = value[:len(value)-1]
|
||||||
|
}
|
||||||
|
return value
|
||||||
|
}
|
||||||
148
internal/loki/client.go
Normal file
148
internal/loki/client.go
Normal file
@ -0,0 +1,148 @@
|
|||||||
|
package loki
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Client struct {
|
||||||
|
baseURL string
|
||||||
|
tenantID string
|
||||||
|
username string
|
||||||
|
password string
|
||||||
|
http *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
func New(baseURL, tenantID, username, password string) *Client {
|
||||||
|
return &Client{baseURL: strings.TrimRight(baseURL, "/"), tenantID: tenantID, username: username, password: password, http: &http.Client{Timeout: 20 * time.Second}}
|
||||||
|
}
|
||||||
|
|
||||||
|
type Match struct {
|
||||||
|
Timestamp time.Time `json:"timestamp"`
|
||||||
|
Line string `json:"line"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type QueryResult struct {
|
||||||
|
Count int `json:"count"`
|
||||||
|
Matches []Match `json:"matches"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) Ready(ctx context.Context) error {
|
||||||
|
req, err := c.request(ctx, http.MethodGet, "/ready", nil)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := c.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode < 200 || resp.StatusCode > 299 {
|
||||||
|
return fmt.Errorf("loki ready failed: status %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) Labels(ctx context.Context) ([]string, error) {
|
||||||
|
var raw labelsResponse
|
||||||
|
if err := c.getJSON(ctx, "/loki/api/v1/labels", nil, &raw); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return raw.Data, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) LabelValues(ctx context.Context, name string) ([]string, error) {
|
||||||
|
if name == "" || strings.Contains(name, "/") {
|
||||||
|
return nil, fmt.Errorf("invalid label name")
|
||||||
|
}
|
||||||
|
var raw labelsResponse
|
||||||
|
if err := c.getJSON(ctx, "/loki/api/v1/label/"+url.PathEscape(name)+"/values", nil, &raw); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return raw.Data, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) QueryRange(ctx context.Context, logql string, since time.Duration, limit int) (QueryResult, error) {
|
||||||
|
if limit <= 0 {
|
||||||
|
limit = 100
|
||||||
|
}
|
||||||
|
values := url.Values{}
|
||||||
|
values.Set("query", logql)
|
||||||
|
values.Set("start", strconv.FormatInt(time.Now().Add(-since).UnixNano(), 10))
|
||||||
|
values.Set("end", strconv.FormatInt(time.Now().UnixNano(), 10))
|
||||||
|
values.Set("limit", strconv.Itoa(limit))
|
||||||
|
var raw response
|
||||||
|
if err := c.getJSON(ctx, "/loki/api/v1/query_range", values, &raw); err != nil {
|
||||||
|
return QueryResult{}, err
|
||||||
|
}
|
||||||
|
var result QueryResult
|
||||||
|
for _, stream := range raw.Data.Result {
|
||||||
|
for _, pair := range stream.Values {
|
||||||
|
if len(pair) != 2 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ns, _ := strconv.ParseInt(pair[0], 10, 64)
|
||||||
|
result.Matches = append(result.Matches, Match{Timestamp: time.Unix(0, ns), Line: pair[1]})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result.Count = len(result.Matches)
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) getJSON(ctx context.Context, path string, values url.Values, out any) error {
|
||||||
|
req, err := c.request(ctx, http.MethodGet, path, values)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := c.http.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode < 200 || resp.StatusCode > 299 {
|
||||||
|
return fmt.Errorf("loki request failed: status %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
return json.NewDecoder(resp.Body).Decode(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) request(ctx context.Context, method, path string, values url.Values) (*http.Request, error) {
|
||||||
|
if c.baseURL == "" {
|
||||||
|
return nil, fmt.Errorf("LOKI_URL is not configured")
|
||||||
|
}
|
||||||
|
target := c.baseURL + path
|
||||||
|
if len(values) > 0 {
|
||||||
|
target += "?" + values.Encode()
|
||||||
|
}
|
||||||
|
req, err := http.NewRequestWithContext(ctx, method, target, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if c.tenantID != "" {
|
||||||
|
req.Header.Set("X-Scope-OrgID", c.tenantID)
|
||||||
|
}
|
||||||
|
if c.username != "" || c.password != "" {
|
||||||
|
req.SetBasicAuth(c.username, c.password)
|
||||||
|
}
|
||||||
|
return req, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type labelsResponse struct {
|
||||||
|
Status string `json:"status"`
|
||||||
|
Data []string `json:"data"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type response struct {
|
||||||
|
Status string `json:"status"`
|
||||||
|
Data struct {
|
||||||
|
Result []struct {
|
||||||
|
Stream map[string]string `json:"stream"`
|
||||||
|
Values [][]string `json:"values"`
|
||||||
|
} `json:"result"`
|
||||||
|
} `json:"data"`
|
||||||
|
}
|
||||||
42
internal/loki/client_test.go
Normal file
42
internal/loki/client_test.go
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
package loki
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestLabelsValuesAndQueryRange(t *testing.T) {
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/loki/api/v1/labels":
|
||||||
|
_, _ = w.Write([]byte(`{"status":"success","data":["service","container"]}`))
|
||||||
|
case "/loki/api/v1/label/service/values":
|
||||||
|
_, _ = w.Write([]byte(`{"status":"success","data":["api"]}`))
|
||||||
|
case "/loki/api/v1/query_range":
|
||||||
|
if !strings.Contains(r.URL.Query().Get("query"), "api") {
|
||||||
|
t.Fatalf("query not passed through: %s", r.URL.RawQuery)
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(`{"status":"success","data":{"result":[{"stream":{"service":"api"},"values":[["1700000000000000000","error token=secret"]]}]}}`))
|
||||||
|
default:
|
||||||
|
http.NotFound(w, r)
|
||||||
|
}
|
||||||
|
}))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
client := New(ts.URL, "", "", "")
|
||||||
|
labels, err := client.Labels(t.Context())
|
||||||
|
if err != nil || len(labels) != 2 {
|
||||||
|
t.Fatalf("labels=%v err=%v", labels, err)
|
||||||
|
}
|
||||||
|
values, err := client.LabelValues(t.Context(), "service")
|
||||||
|
if err != nil || values[0] != "api" {
|
||||||
|
t.Fatalf("values=%v err=%v", values, err)
|
||||||
|
}
|
||||||
|
result, err := client.QueryRange(t.Context(), `{service="api"}`, time.Minute, 10)
|
||||||
|
if err != nil || result.Count != 1 {
|
||||||
|
t.Fatalf("result=%#v err=%v", result, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
152
internal/rules/engine.go
Normal file
152
internal/rules/engine.go
Normal file
@ -0,0 +1,152 @@
|
|||||||
|
package rules
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/actions"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Store interface {
|
||||||
|
Load() (store.Data, error)
|
||||||
|
UpdateRule(id string, fn func(*store.Rule)) error
|
||||||
|
AddIncident(incident store.Incident) error
|
||||||
|
}
|
||||||
|
|
||||||
|
type Engine struct {
|
||||||
|
store Store
|
||||||
|
loki *loki.Client
|
||||||
|
analyzer analysis.Analyzer
|
||||||
|
actions *actions.Runner
|
||||||
|
dispatcher *alerts.Dispatcher
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewEngine(s Store, lokiClient *loki.Client, analyzer analysis.Analyzer, actionRunner *actions.Runner, dispatcher *alerts.Dispatcher) *Engine {
|
||||||
|
return &Engine{store: s, loki: lokiClient, analyzer: analyzer, actions: actionRunner, dispatcher: dispatcher}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Engine) Start(ctx context.Context, interval time.Duration) {
|
||||||
|
if interval <= 0 {
|
||||||
|
interval = time.Minute
|
||||||
|
}
|
||||||
|
ticker := time.NewTicker(interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
e.CheckAll(ctx)
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Engine) CheckAll(ctx context.Context) {
|
||||||
|
data, err := e.store.Load()
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("load rules failed: %v", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
channels := map[string]store.AlertChannel{}
|
||||||
|
for _, channel := range data.AlertChannels {
|
||||||
|
channels[channel.ID] = channel
|
||||||
|
}
|
||||||
|
for _, rule := range data.Rules {
|
||||||
|
if !rule.Enabled {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := e.checkRule(ctx, rule, channels); err != nil {
|
||||||
|
log.Printf("rule %q check failed: %v", rule.Name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Engine) checkRule(ctx context.Context, rule store.Rule, channels map[string]store.AlertChannel) error {
|
||||||
|
window, err := time.ParseDuration(rule.Window)
|
||||||
|
if err != nil {
|
||||||
|
window = 5 * time.Minute
|
||||||
|
}
|
||||||
|
result, err := e.loki.QueryRange(ctx, rule.LogQL, window, 200)
|
||||||
|
if err != nil {
|
||||||
|
e.record(rule.ID, 0, err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if result.Count < rule.Threshold {
|
||||||
|
return e.record(rule.ID, result.Count, nil)
|
||||||
|
}
|
||||||
|
now := time.Now().UTC()
|
||||||
|
cooldown, err := time.ParseDuration(rule.Cooldown)
|
||||||
|
if err != nil || cooldown <= 0 {
|
||||||
|
cooldown = time.Hour
|
||||||
|
}
|
||||||
|
if !rule.LastAlertedAt.IsZero() && now.Sub(rule.LastAlertedAt) < cooldown {
|
||||||
|
return e.store.UpdateRule(rule.ID, func(r *store.Rule) {
|
||||||
|
r.LastCheckedAt = now
|
||||||
|
r.LastMatchedAt = now
|
||||||
|
r.LastMatchCount = result.Count
|
||||||
|
r.SuppressedCount++
|
||||||
|
r.LastError = ""
|
||||||
|
})
|
||||||
|
}
|
||||||
|
finding, err := e.analyzer.Analyze(ctx, rule, result.Matches)
|
||||||
|
if err != nil {
|
||||||
|
e.record(rule.ID, result.Count, err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
selected := make([]store.AlertChannel, 0, len(rule.AlertChannels))
|
||||||
|
for _, id := range rule.AlertChannels {
|
||||||
|
if channel, ok := channels[id]; ok {
|
||||||
|
selected = append(selected, channel)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
alertResults, alertErr := e.dispatcher.Send(ctx, rule, finding, selected)
|
||||||
|
var alertEvidence []string
|
||||||
|
for _, result := range alertResults {
|
||||||
|
if result.ChannelID == "" {
|
||||||
|
alertEvidence = append(alertEvidence, result.Detail)
|
||||||
|
} else {
|
||||||
|
alertEvidence = append(alertEvidence, result.ChannelID+": "+result.Detail)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var actionEvidence []string
|
||||||
|
for _, action := range rule.Actions {
|
||||||
|
result, err := e.actions.Run(ctx, rule, action)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("action %q for rule %q failed: %v", action.Type, rule.Name, err)
|
||||||
|
actionEvidence = append(actionEvidence, action.Type+": "+err.Error())
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
actionEvidence = append(actionEvidence, result.Action+": "+result.Detail)
|
||||||
|
}
|
||||||
|
incident := store.Incident{RuleID: rule.ID, RuleName: rule.Name, Severity: rule.Severity, Count: result.Count, Summary: finding.Summary, AlertResults: alertEvidence, RemediationEvidence: actionEvidence, CreatedAt: now}
|
||||||
|
if err := e.store.AddIncident(incident); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := e.store.UpdateRule(rule.ID, func(r *store.Rule) {
|
||||||
|
r.LastCheckedAt = now
|
||||||
|
r.LastMatchedAt = now
|
||||||
|
r.LastAlertedAt = now
|
||||||
|
r.LastMatchCount = result.Count
|
||||||
|
r.LastError = ""
|
||||||
|
}); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return alertErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Engine) record(id string, count int, err error) error {
|
||||||
|
return e.store.UpdateRule(id, func(r *store.Rule) {
|
||||||
|
r.LastCheckedAt = time.Now().UTC()
|
||||||
|
r.LastMatchCount = count
|
||||||
|
if err != nil {
|
||||||
|
r.LastError = err.Error()
|
||||||
|
} else {
|
||||||
|
r.LastError = ""
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
42
internal/rules/engine_test.go
Normal file
42
internal/rules/engine_test.go
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
package rules
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/actions"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestCooldownSuppressesDuplicateIncident(t *testing.T) {
|
||||||
|
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_, _ = w.Write([]byte(`{"status":"success","data":{"result":[{"stream":{},"values":[["1700000000000000000","error"]]}]}}`))
|
||||||
|
}))
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
s := store.NewFileStore(t.TempDir() + "/config.json")
|
||||||
|
if err := s.UpsertRule(store.Rule{Name: "Errors", Enabled: true, LogQL: `{service="api"}`, Threshold: 1, Window: "5m", Cooldown: "1h", Actions: []store.Action{{Type: "record_recommendation", Enabled: true}}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
engine := NewEngine(s, loki.New(ts.URL, "", "", ""), analysis.NoopAnalyzer{}, actions.NewRunner(true), alerts.NewDispatcher())
|
||||||
|
engine.CheckAll(t.Context())
|
||||||
|
engine.CheckAll(t.Context())
|
||||||
|
data, err := s.Load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(data.Incidents) != 1 {
|
||||||
|
t.Fatalf("expected one incident, got %d", len(data.Incidents))
|
||||||
|
}
|
||||||
|
if data.Rules[0].SuppressedCount != 1 {
|
||||||
|
t.Fatalf("expected suppressed count 1, got %d", data.Rules[0].SuppressedCount)
|
||||||
|
}
|
||||||
|
if time.Since(data.Rules[0].LastAlertedAt) > time.Minute {
|
||||||
|
t.Fatalf("last alerted not set: %s", data.Rules[0].LastAlertedAt)
|
||||||
|
}
|
||||||
|
}
|
||||||
51
internal/server/middleware.go
Normal file
51
internal/server/middleware.go
Normal file
@ -0,0 +1,51 @@
|
|||||||
|
package server
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/subtle"
|
||||||
|
"encoding/json"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func writeJSON(w http.ResponseWriter, v any) {
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
_ = json.NewEncoder(w).Encode(v)
|
||||||
|
}
|
||||||
|
|
||||||
|
func requestLog(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
start := time.Now()
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
log.Printf("%s %s %s", r.Method, r.URL.Path, time.Since(start).Round(time.Millisecond))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func securityHeaders(next http.Handler) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||||
|
w.Header().Set("Referrer-Policy", "no-referrer")
|
||||||
|
w.Header().Set("Content-Security-Policy", "default-src 'self'; style-src 'self' 'unsafe-inline'; form-action 'self'")
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func basicAuth(next http.Handler, username, password string) http.Handler {
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/healthz" {
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if username == "" || password == "" {
|
||||||
|
http.Error(w, "basic auth is not configured", http.StatusServiceUnavailable)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
gotUser, gotPass, ok := r.BasicAuth()
|
||||||
|
if !ok || subtle.ConstantTimeCompare([]byte(gotUser), []byte(username)) != 1 || subtle.ConstantTimeCompare([]byte(gotPass), []byte(password)) != 1 {
|
||||||
|
w.Header().Set("WWW-Authenticate", `Basic realm="log-guardian"`)
|
||||||
|
http.Error(w, "unauthorized", http.StatusUnauthorized)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
next.ServeHTTP(w, r)
|
||||||
|
})
|
||||||
|
}
|
||||||
34
internal/server/middleware_test.go
Normal file
34
internal/server/middleware_test.go
Normal file
@ -0,0 +1,34 @@
|
|||||||
|
package server
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestBasicAuthAllowsHealthzWithoutCredentials(t *testing.T) {
|
||||||
|
h := basicAuth(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) }), "admin", "secret")
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusNoContent {
|
||||||
|
t.Fatalf("got %d", rr.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBasicAuthProtectsUI(t *testing.T) {
|
||||||
|
h := basicAuth(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) }), "admin", "secret")
|
||||||
|
req := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||||
|
rr := httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusUnauthorized {
|
||||||
|
t.Fatalf("got %d", rr.Code)
|
||||||
|
}
|
||||||
|
req = httptest.NewRequest(http.MethodGet, "/", nil)
|
||||||
|
req.SetBasicAuth("admin", "secret")
|
||||||
|
rr = httptest.NewRecorder()
|
||||||
|
h.ServeHTTP(rr, req)
|
||||||
|
if rr.Code != http.StatusNoContent {
|
||||||
|
t.Fatalf("got %d", rr.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
202
internal/server/server.go
Normal file
202
internal/server/server.go
Normal file
@ -0,0 +1,202 @@
|
|||||||
|
package server
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"html/template"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/rules"
|
||||||
|
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Server struct {
|
||||||
|
store *store.FileStore
|
||||||
|
engine *rules.Engine
|
||||||
|
loki *loki.Client
|
||||||
|
dispatcher *alerts.Dispatcher
|
||||||
|
authUsername string
|
||||||
|
authPassword string
|
||||||
|
}
|
||||||
|
|
||||||
|
func New(s *store.FileStore, engine *rules.Engine, lokiClient *loki.Client, dispatcher *alerts.Dispatcher, authUsername, authPassword string) *Server {
|
||||||
|
return &Server{store: s, engine: engine, loki: lokiClient, dispatcher: dispatcher, authUsername: authUsername, authPassword: authPassword}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) Routes() http.Handler {
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.HandleFunc("GET /healthz", s.health)
|
||||||
|
mux.HandleFunc("GET /", s.index)
|
||||||
|
mux.HandleFunc("POST /rules", s.saveRule)
|
||||||
|
mux.HandleFunc("POST /check", s.checkNow)
|
||||||
|
mux.HandleFunc("POST /channels", s.saveChannel)
|
||||||
|
mux.HandleFunc("POST /channels/delete", s.deleteChannel)
|
||||||
|
mux.HandleFunc("POST /channels/test", s.testChannel)
|
||||||
|
mux.HandleFunc("GET /loki/labels", s.lokiLabels)
|
||||||
|
mux.HandleFunc("GET /loki/label-values", s.lokiLabelValues)
|
||||||
|
mux.HandleFunc("POST /loki/query-test", s.lokiQueryTest)
|
||||||
|
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServer(http.Dir("web/static"))))
|
||||||
|
return securityHeaders(requestLog(basicAuth(mux, s.authUsername, s.authPassword)))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) health(w http.ResponseWriter, r *http.Request) {
|
||||||
|
writeJSON(w, map[string]string{"status": "ok", "service": "log-guardian"})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) index(w http.ResponseWriter, r *http.Request) {
|
||||||
|
data, err := s.store.Load()
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
render(w, data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) saveRule(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
threshold, _ := strconv.Atoi(r.FormValue("threshold"))
|
||||||
|
rule := store.Rule{ID: strings.TrimSpace(r.FormValue("id")), Name: strings.TrimSpace(r.FormValue("name")), Enabled: r.FormValue("enabled") == "on", LogQL: strings.TrimSpace(r.FormValue("logql")), Severity: strings.TrimSpace(r.FormValue("severity")), Threshold: threshold, Window: strings.TrimSpace(r.FormValue("window")), Cooldown: strings.TrimSpace(r.FormValue("cooldown")), AnalysisEnabled: r.FormValue("analysis_enabled") == "on", AlertChannels: r.Form["alert_channels"]}
|
||||||
|
if rule.Name == "" || rule.LogQL == "" {
|
||||||
|
http.Error(w, "name and LogQL are required", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if rule.Severity == "" {
|
||||||
|
rule.Severity = "warning"
|
||||||
|
}
|
||||||
|
if err := s.store.UpsertRule(rule); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
http.Redirect(w, r, "/", http.StatusSeeOther)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) saveChannel(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
channel := store.AlertChannel{ID: strings.TrimSpace(r.FormValue("id")), Name: strings.TrimSpace(r.FormValue("name")), Type: strings.TrimSpace(r.FormValue("type")), Enabled: r.FormValue("enabled") == "on", Params: map[string]string{}}
|
||||||
|
for _, key := range []string{"server_url", "topic", "token", "priority", "tags", "url", "api_url", "email"} {
|
||||||
|
if value := strings.TrimSpace(r.FormValue(key)); value != "" {
|
||||||
|
channel.Params[key] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if channel.Name == "" || channel.Type == "" {
|
||||||
|
http.Error(w, "name and type are required", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := s.store.UpsertAlertChannel(channel); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
http.Redirect(w, r, "/", http.StatusSeeOther)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) deleteChannel(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := s.store.DeleteAlertChannel(r.FormValue("id")); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
http.Redirect(w, r, "/", http.StatusSeeOther)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) testChannel(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
data, err := s.store.Load()
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, channel := range data.AlertChannels {
|
||||||
|
if channel.ID == r.FormValue("id") {
|
||||||
|
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if err := s.dispatcher.Test(ctx, channel); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, map[string]string{"status": "sent"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
http.Error(w, "channel not found", http.StatusNotFound)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) checkNow(w http.ResponseWriter, r *http.Request) {
|
||||||
|
s.engine.CheckAll(context.Background())
|
||||||
|
http.Redirect(w, r, "/", http.StatusSeeOther)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) lokiLabels(w http.ResponseWriter, r *http.Request) {
|
||||||
|
labels, err := s.loki.Labels(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, map[string]any{"labels": labels})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) lokiLabelValues(w http.ResponseWriter, r *http.Request) {
|
||||||
|
values, err := s.loki.LabelValues(r.Context(), r.URL.Query().Get("name"))
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, map[string]any{"values": values})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) lokiQueryTest(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
window, err := time.ParseDuration(r.FormValue("window"))
|
||||||
|
if err != nil {
|
||||||
|
window = 5 * time.Minute
|
||||||
|
}
|
||||||
|
result, err := s.loki.QueryRange(r.Context(), strings.TrimSpace(r.FormValue("logql")), window, 20)
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(result.Matches) > 5 {
|
||||||
|
result.Matches = result.Matches[:5]
|
||||||
|
}
|
||||||
|
writeJSON(w, result)
|
||||||
|
}
|
||||||
|
|
||||||
|
var tmpl = template.Must(template.New("index").Funcs(template.FuncMap{"secretState": secretState}).Parse(`<!doctype html>
|
||||||
|
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Log Guardian</title><link rel="stylesheet" href="/static/app.css"></head>
|
||||||
|
<body><main class="shell"><header><h1>Log Guardian</h1><p>Loki-backed log monitoring, alerting, and guarded remediation.</p></header>
|
||||||
|
<section class="card"><h2>Loki diagnostics</h2><p>Use authenticated endpoints: <code>/loki/labels</code>, <code>/loki/label-values?name=service</code>.</p><form method="post" action="/loki/query-test" class="grid"><label class="wide">Test LogQL <input name="logql" placeholder='{service="api"} |= "error"'></label><label>Window <input name="window" value="5m"></label><button type="submit">Return JSON sample</button></form></section>
|
||||||
|
<section class="card"><h2>Add or replace rule</h2><form method="post" action="/rules" class="grid"><label>Name <input name="name" required placeholder="API errors"></label><label>ID <input name="id" placeholder="optional stable id"></label><label>Severity <input name="severity" value="warning"></label><label>Window <input name="window" value="5m"></label><label>Cooldown <input name="cooldown" value="1h"></label><label>Threshold <input name="threshold" type="number" min="1" value="1"></label><label class="wide">LogQL <input name="logql" required placeholder='{service="api"} |= "error"'></label><fieldset class="wide"><legend>Alert channels</legend>{{range .AlertChannels}}<label class="check"><input type="checkbox" name="alert_channels" value="{{.ID}}"> {{.Name}} ({{.Type}})</label>{{else}}<span class="muted">No channels configured.</span>{{end}}</fieldset><label class="check"><input type="checkbox" name="enabled" checked> Enabled</label><label class="check"><input type="checkbox" name="analysis_enabled"> AI analysis</label><button type="submit">Save rule</button></form></section>
|
||||||
|
<section class="card"><h2>Alert channels</h2><form method="post" action="/channels" class="grid"><label>Name <input name="name" required></label><label>ID <input name="id" placeholder="optional stable id"></label><label>Type <select name="type"><option value="ntfy">ntfy</option><option value="gotify">gotify (scaffold)</option><option value="generic_webhook">generic webhook (scaffold)</option><option value="smtp">smtp (scaffold)</option></select></label><label class="check"><input type="checkbox" name="enabled" checked> Enabled</label><label>Server URL <input name="server_url" placeholder="https://ntfy.example.com"></label><label>Topic <input name="topic"></label><label>Token <input name="token" type="password" placeholder="leave blank to preserve"></label><label>Priority <input name="priority" placeholder="default, high, urgent"></label><label>Tags <input name="tags" placeholder="warning,rotating_light"></label><button type="submit">Save channel</button></form><table><thead><tr><th>Name</th><th>Type</th><th>Status</th><th>Secret</th><th>Actions</th></tr></thead><tbody>{{range .AlertChannels}}<tr><td>{{.Name}}</td><td>{{.Type}}</td><td>{{if .Enabled}}enabled{{else}}disabled{{end}}</td><td>{{secretState .Params}}</td><td><form method="post" action="/channels/test" class="inline"><input type="hidden" name="id" value="{{.ID}}"><button>Test</button></form><form method="post" action="/channels/delete" class="inline"><input type="hidden" name="id" value="{{.ID}}"><button>Delete</button></form></td></tr>{{else}}<tr><td colspan="5" class="muted">No alert channels configured.</td></tr>{{end}}</tbody></table></section>
|
||||||
|
<section class="card"><div class="row"><h2>Rules</h2><form method="post" action="/check"><button type="submit">Check now</button></form></div><table><thead><tr><th>Name</th><th>LogQL</th><th>Cooldown</th><th>Last count</th><th>Suppressed</th><th>Status</th></tr></thead><tbody>{{range .Rules}}<tr><td>{{.Name}}{{if not .Enabled}} <span class="muted">disabled</span>{{end}}</td><td><code>{{.LogQL}}</code></td><td>{{.Cooldown}}</td><td>{{.LastMatchCount}}</td><td>{{.SuppressedCount}}</td><td>{{if .LastError}}<span class="error">{{.LastError}}</span>{{else}}checked {{.LastCheckedAt}}{{end}}</td></tr>{{else}}<tr><td colspan="6" class="muted">No rules configured yet.</td></tr>{{end}}</tbody></table></section>
|
||||||
|
<section class="card"><h2>Recent incidents</h2><table><thead><tr><th>Time</th><th>Rule</th><th>Severity</th><th>Summary</th><th>Evidence</th></tr></thead><tbody>{{range .Incidents}}<tr><td>{{.CreatedAt}}</td><td>{{.RuleName}}</td><td>{{.Severity}}</td><td>{{.Summary}}</td><td>{{range .AlertResults}}<div>{{.}}</div>{{end}}{{range .RemediationEvidence}}<div>{{.}}</div>{{end}}</td></tr>{{else}}<tr><td colspan="5" class="muted">No incidents recorded.</td></tr>{{end}}</tbody></table></section>
|
||||||
|
</main></body></html>`))
|
||||||
|
|
||||||
|
func secretState(params map[string]string) string {
|
||||||
|
if params["token"] != "" || params["password"] != "" || params["secret"] != "" {
|
||||||
|
return "configured"
|
||||||
|
}
|
||||||
|
return "not configured"
|
||||||
|
}
|
||||||
|
|
||||||
|
func render(w http.ResponseWriter, data store.Data) {
|
||||||
|
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||||
|
_ = tmpl.Execute(w, data)
|
||||||
|
}
|
||||||
260
internal/store/store.go
Normal file
260
internal/store/store.go
Normal file
@ -0,0 +1,260 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Rule struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
LogQL string `json:"logql"`
|
||||||
|
Severity string `json:"severity"`
|
||||||
|
Threshold int `json:"threshold"`
|
||||||
|
Window string `json:"window"`
|
||||||
|
Cooldown string `json:"cooldown,omitempty"`
|
||||||
|
AlertChannels []string `json:"alert_channels,omitempty"`
|
||||||
|
AnalysisEnabled bool `json:"analysis_enabled"`
|
||||||
|
Actions []Action `json:"actions,omitempty"`
|
||||||
|
LastCheckedAt time.Time `json:"last_checked_at,omitempty"`
|
||||||
|
LastMatchedAt time.Time `json:"last_matched_at,omitempty"`
|
||||||
|
LastAlertedAt time.Time `json:"last_alerted_at,omitempty"`
|
||||||
|
LastMatchCount int `json:"last_match_count,omitempty"`
|
||||||
|
SuppressedCount int `json:"suppressed_count,omitempty"`
|
||||||
|
LastError string `json:"last_error,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Action struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
DryRun *bool `json:"dry_run,omitempty"`
|
||||||
|
Params map[string]string `json:"params,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type AlertChannel struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Type string `json:"type"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
Params map[string]string `json:"params,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Incident struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
RuleID string `json:"rule_id"`
|
||||||
|
RuleName string `json:"rule_name"`
|
||||||
|
Severity string `json:"severity"`
|
||||||
|
Count int `json:"count"`
|
||||||
|
Summary string `json:"summary"`
|
||||||
|
AlertResults []string `json:"alert_results,omitempty"`
|
||||||
|
RemediationEvidence []string `json:"remediation_evidence,omitempty"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Data struct {
|
||||||
|
Rules []Rule `json:"rules"`
|
||||||
|
AlertChannels []AlertChannel `json:"alert_channels"`
|
||||||
|
Incidents []Incident `json:"incidents"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type FileStore struct {
|
||||||
|
path string
|
||||||
|
mu sync.Mutex
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewFileStore(path string) *FileStore { return &FileStore{path: path} }
|
||||||
|
|
||||||
|
func (s *FileStore) Load() (Data, error) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.loadLocked()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) Save(data Data) error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) UpsertRule(rule Rule) error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
data, err := s.loadLocked()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if rule.ID == "" {
|
||||||
|
rule.ID = slug(rule.Name)
|
||||||
|
}
|
||||||
|
if rule.ID == "" {
|
||||||
|
return errors.New("rule id or name is required")
|
||||||
|
}
|
||||||
|
if rule.Threshold <= 0 {
|
||||||
|
rule.Threshold = 1
|
||||||
|
}
|
||||||
|
if rule.Window == "" {
|
||||||
|
rule.Window = "5m"
|
||||||
|
}
|
||||||
|
if rule.Cooldown == "" {
|
||||||
|
rule.Cooldown = "1h"
|
||||||
|
}
|
||||||
|
for i := range data.Rules {
|
||||||
|
if data.Rules[i].ID == rule.ID {
|
||||||
|
rule.LastCheckedAt = data.Rules[i].LastCheckedAt
|
||||||
|
rule.LastMatchedAt = data.Rules[i].LastMatchedAt
|
||||||
|
rule.LastAlertedAt = data.Rules[i].LastAlertedAt
|
||||||
|
rule.LastMatchCount = data.Rules[i].LastMatchCount
|
||||||
|
rule.SuppressedCount = data.Rules[i].SuppressedCount
|
||||||
|
rule.LastError = data.Rules[i].LastError
|
||||||
|
data.Rules[i] = rule
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
data.Rules = append(data.Rules, rule)
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) UpsertAlertChannel(channel AlertChannel) error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
data, err := s.loadLocked()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if channel.ID == "" {
|
||||||
|
channel.ID = slug(channel.Name)
|
||||||
|
}
|
||||||
|
if channel.ID == "" {
|
||||||
|
return errors.New("channel id or name is required")
|
||||||
|
}
|
||||||
|
if channel.Params == nil {
|
||||||
|
channel.Params = map[string]string{}
|
||||||
|
}
|
||||||
|
for i := range data.AlertChannels {
|
||||||
|
if data.AlertChannels[i].ID == channel.ID {
|
||||||
|
preserveSecrets(data.AlertChannels[i].Params, channel.Params)
|
||||||
|
data.AlertChannels[i] = channel
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
data.AlertChannels = append(data.AlertChannels, channel)
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func preserveSecrets(oldParams, newParams map[string]string) {
|
||||||
|
for _, key := range []string{"token", "password", "secret", "authorization"} {
|
||||||
|
if newParams[key] == "" && oldParams[key] != "" {
|
||||||
|
newParams[key] = oldParams[key]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) DeleteAlertChannel(id string) error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
data, err := s.loadLocked()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
channels := data.AlertChannels[:0]
|
||||||
|
for _, channel := range data.AlertChannels {
|
||||||
|
if channel.ID != id {
|
||||||
|
channels = append(channels, channel)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
data.AlertChannels = channels
|
||||||
|
for i := range data.Rules {
|
||||||
|
ids := data.Rules[i].AlertChannels[:0]
|
||||||
|
for _, channelID := range data.Rules[i].AlertChannels {
|
||||||
|
if channelID != id {
|
||||||
|
ids = append(ids, channelID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
data.Rules[i].AlertChannels = ids
|
||||||
|
}
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) UpdateRule(id string, fn func(*Rule)) error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
data, err := s.loadLocked()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for i := range data.Rules {
|
||||||
|
if data.Rules[i].ID == id {
|
||||||
|
fn(&data.Rules[i])
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return errors.New("rule not found")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) AddIncident(incident Incident) error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
data, err := s.loadLocked()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if incident.ID == "" {
|
||||||
|
incident.ID = slug(incident.RuleName) + "-" + incident.CreatedAt.Format("20060102150405")
|
||||||
|
}
|
||||||
|
data.Incidents = append([]Incident{incident}, data.Incidents...)
|
||||||
|
if len(data.Incidents) > 200 {
|
||||||
|
data.Incidents = data.Incidents[:200]
|
||||||
|
}
|
||||||
|
return s.saveLocked(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) loadLocked() (Data, error) {
|
||||||
|
var data Data
|
||||||
|
contents, err := os.ReadFile(s.path)
|
||||||
|
if errors.Is(err, os.ErrNotExist) {
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return data, err
|
||||||
|
}
|
||||||
|
if len(contents) == 0 {
|
||||||
|
return data, nil
|
||||||
|
}
|
||||||
|
return data, json.Unmarshal(contents, &data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *FileStore) saveLocked(data Data) error {
|
||||||
|
if err := os.MkdirAll(filepath.Dir(s.path), 0o700); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
contents, err := json.MarshalIndent(data, "", " ")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.WriteFile(s.path, append(contents, '\n'), 0o600)
|
||||||
|
}
|
||||||
|
|
||||||
|
func slug(value string) string {
|
||||||
|
value = strings.ToLower(strings.TrimSpace(value))
|
||||||
|
var b strings.Builder
|
||||||
|
lastDash := false
|
||||||
|
for _, r := range value {
|
||||||
|
ok := r >= 'a' && r <= 'z' || r >= '0' && r <= '9'
|
||||||
|
if ok {
|
||||||
|
b.WriteRune(r)
|
||||||
|
lastDash = false
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !lastDash {
|
||||||
|
b.WriteByte('-')
|
||||||
|
lastDash = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return strings.Trim(b.String(), "-")
|
||||||
|
}
|
||||||
38
internal/store/store_test.go
Normal file
38
internal/store/store_test.go
Normal file
@ -0,0 +1,38 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestUpsertRuleDefaults(t *testing.T) {
|
||||||
|
s := NewFileStore(t.TempDir() + "/config.json")
|
||||||
|
if err := s.UpsertRule(Rule{Name: "API Errors", LogQL: `{service="api"} |= "error"`}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
data, err := s.Load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(data.Rules) != 1 {
|
||||||
|
t.Fatalf("expected one rule, got %d", len(data.Rules))
|
||||||
|
}
|
||||||
|
rule := data.Rules[0]
|
||||||
|
if rule.ID != "api-errors" || rule.Threshold != 1 || rule.Window != "5m" || rule.Cooldown != "1h" {
|
||||||
|
t.Fatalf("unexpected defaults: %#v", rule)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpsertAlertChannelPreservesSecret(t *testing.T) {
|
||||||
|
s := NewFileStore(t.TempDir() + "/config.json")
|
||||||
|
if err := s.UpsertAlertChannel(AlertChannel{ID: "ntfy", Name: "ntfy", Type: "ntfy", Enabled: true, Params: map[string]string{"token": "secret", "topic": "alerts"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.UpsertAlertChannel(AlertChannel{ID: "ntfy", Name: "ntfy", Type: "ntfy", Enabled: true, Params: map[string]string{"topic": "alerts2"}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
data, err := s.Load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := data.AlertChannels[0].Params["token"]; got != "secret" {
|
||||||
|
t.Fatalf("secret was not preserved: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
285
project-docs/design/phase-1-plan.md
Normal file
285
project-docs/design/phase-1-plan.md
Normal file
@ -0,0 +1,285 @@
|
|||||||
|
# Phase 1 Plan
|
||||||
|
|
||||||
|
Goal: make log monitoring useful in production without enabling dangerous automation.
|
||||||
|
|
||||||
|
Phase 1 targets a production Portainer deployment of Log Guardian. Loki will likely run on a separate server and use an object-storage backend, probably MinIO backed by NAS storage. Log Guardian must remain safe-by-default: authenticated UI, real alert delivery, incident cooldowns, Loki validation tooling, dry-run remediation evidence, and local-only AI analysis.
|
||||||
|
|
||||||
|
## Decisions made
|
||||||
|
|
||||||
|
- Deployment target: production Portainer.
|
||||||
|
- Loki location: separate server from Log Guardian.
|
||||||
|
- Loki storage direction: object storage via MinIO, with MinIO data on NAS-backed storage.
|
||||||
|
- UI auth: single admin user using basic auth.
|
||||||
|
- Health endpoint: `/healthz` remains unauthenticated.
|
||||||
|
- First alert channel: self-hosted ntfy.
|
||||||
|
- Additional alert channel types should be supported/configurable from UI, not ntfy-only.
|
||||||
|
- Incident cooldown: per rule, default `1h`.
|
||||||
|
- AI: local-only Ollama is approved; no external/cloud AI providers.
|
||||||
|
- Remediation: Phase 1 should implement permission modeling and dry-run evidence, not dangerous automatic actions.
|
||||||
|
- Phase 1 completion requires Docker build and real alert delivery validation.
|
||||||
|
|
||||||
|
## 1. Production deployment architecture
|
||||||
|
|
||||||
|
### Log Guardian
|
||||||
|
|
||||||
|
Run Log Guardian as a Portainer-managed service/container.
|
||||||
|
|
||||||
|
Required mounts/config:
|
||||||
|
|
||||||
|
- Persistent local config/data file mount for `LOG_GUARDIAN_DATA_PATH`.
|
||||||
|
- Environment configuration for Loki endpoint, auth, alert defaults, and optional Ollama endpoint.
|
||||||
|
- Network path to the remote Loki server.
|
||||||
|
- Optional network path to self-hosted ntfy, Gotify, SMTP relay, webhook targets, and Ollama.
|
||||||
|
|
||||||
|
### Loki
|
||||||
|
|
||||||
|
Loki will run on a different server. Recommended production direction:
|
||||||
|
|
||||||
|
- Loki single-binary or simple scalable deployment depending on expected volume.
|
||||||
|
- Object storage backend: S3-compatible MinIO.
|
||||||
|
- MinIO data stored on NAS-backed storage.
|
||||||
|
- Retention configured explicitly.
|
||||||
|
- Promtail or Grafana Alloy deployed on every Docker/Portainer host that needs logs collected.
|
||||||
|
|
||||||
|
Recommended initial storage sizing:
|
||||||
|
|
||||||
|
- Start with 50 GB usable object storage for modest private infrastructure.
|
||||||
|
- Use 14-day retention initially.
|
||||||
|
- Monitor ingestion and object growth for 7 days before increasing retention.
|
||||||
|
- Increase to 100-250 GB+ if services are noisy or retention must be longer.
|
||||||
|
|
||||||
|
Planning assumptions to validate:
|
||||||
|
|
||||||
|
- NAS latency is acceptable for MinIO/Loki workload.
|
||||||
|
- MinIO has backups/snapshots appropriate for desired retention.
|
||||||
|
- Loki retention deletes old chunks/index data correctly.
|
||||||
|
- Log Guardian can reach Loki over a private network/VPN only.
|
||||||
|
|
||||||
|
## 2. Loki ingestion, label discovery, and validation
|
||||||
|
|
||||||
|
Because actual labels are not known yet, Phase 1 must include label discovery and query testing.
|
||||||
|
|
||||||
|
### Collection strategy
|
||||||
|
|
||||||
|
Use Promtail or Grafana Alloy on Docker/Portainer hosts. Target labels should include stable, low-cardinality dimensions such as:
|
||||||
|
|
||||||
|
- `host`
|
||||||
|
- `container`
|
||||||
|
- `service`
|
||||||
|
- `stack`
|
||||||
|
- `stream`
|
||||||
|
- `compose_project` or equivalent
|
||||||
|
- `compose_service` or equivalent
|
||||||
|
|
||||||
|
Avoid high-cardinality labels, including request IDs, user IDs, IP addresses, paths, error messages, or raw log content.
|
||||||
|
|
||||||
|
### Log Guardian Loki diagnostics
|
||||||
|
|
||||||
|
Add authenticated UI/API support for:
|
||||||
|
|
||||||
|
- Loki readiness/status check.
|
||||||
|
- List labels from `/loki/api/v1/labels`.
|
||||||
|
- List values for a selected label from `/loki/api/v1/label/{name}/values`.
|
||||||
|
- Test a LogQL query before saving a rule.
|
||||||
|
- Show count and sanitized sample lines for a test query.
|
||||||
|
|
||||||
|
Implementation notes:
|
||||||
|
|
||||||
|
- Limit query time range and result count.
|
||||||
|
- Do not log raw sample payloads by default.
|
||||||
|
- Redact obvious secrets from displayed query samples where practical.
|
||||||
|
- Handle Loki tenant ID and basic auth from existing Loki config.
|
||||||
|
|
||||||
|
## 3. UI authentication
|
||||||
|
|
||||||
|
Add single-admin basic auth.
|
||||||
|
|
||||||
|
Configuration:
|
||||||
|
|
||||||
|
- `LOG_GUARDIAN_AUTH_USERNAME`
|
||||||
|
- `LOG_GUARDIAN_AUTH_PASSWORD`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- `/healthz` is unauthenticated.
|
||||||
|
- All other UI/API routes require auth.
|
||||||
|
- Use constant-time comparison for credentials.
|
||||||
|
- Do not log usernames/passwords or auth headers.
|
||||||
|
- If credentials are unset, fail closed in production or clearly mark auth as disabled only for local development. Prefer fail closed for Portainer deployment.
|
||||||
|
|
||||||
|
## 4. Alert channels
|
||||||
|
|
||||||
|
Alert channels must be configurable from the UI and stored in local configuration. Secrets need careful handling.
|
||||||
|
|
||||||
|
### Required first channel: self-hosted ntfy
|
||||||
|
|
||||||
|
Support fields:
|
||||||
|
|
||||||
|
- Name
|
||||||
|
- Enabled
|
||||||
|
- Server URL
|
||||||
|
- Topic
|
||||||
|
- Optional bearer token or token file reference
|
||||||
|
- Priority
|
||||||
|
- Tags
|
||||||
|
|
||||||
|
Validation:
|
||||||
|
|
||||||
|
- Send a test notification from the UI.
|
||||||
|
- Record delivery result without logging token values.
|
||||||
|
|
||||||
|
### Additional Phase 1 channel types
|
||||||
|
|
||||||
|
Implement the data model and UI so multiple channel types are possible. Preferred Phase 1 support order:
|
||||||
|
|
||||||
|
1. `ntfy` - implement real delivery first.
|
||||||
|
2. `gotify` - real delivery if time allows; token-based API is straightforward.
|
||||||
|
3. `generic_webhook` - useful but must avoid leaking secrets; support explicit headers carefully.
|
||||||
|
4. `smtp` - useful, but can be deferred if it complicates TLS/auth validation.
|
||||||
|
|
||||||
|
Minimum acceptable Phase 1 scope:
|
||||||
|
|
||||||
|
- UI can create/edit/delete alert channels.
|
||||||
|
- UI can attach channels to rules.
|
||||||
|
- ntfy real delivery works.
|
||||||
|
- Other channel types may be present as disabled/scaffolded if not implemented, but the UI should clearly show unsupported status.
|
||||||
|
|
||||||
|
Secret handling:
|
||||||
|
|
||||||
|
- Do not print secrets in logs.
|
||||||
|
- Do not render stored secret values back into forms; show only `configured`/`not configured`.
|
||||||
|
- Prefer environment variable or mounted file references for sensitive channel tokens when possible.
|
||||||
|
- If secrets are stored in local JSON, the file must be written with `0600` permissions and documented as sensitive.
|
||||||
|
|
||||||
|
## 5. Rule evaluation and incident cooldown
|
||||||
|
|
||||||
|
Add per-rule cooldown.
|
||||||
|
|
||||||
|
Rule field:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"cooldown": "1h"
|
||||||
|
```
|
||||||
|
|
||||||
|
Default behavior:
|
||||||
|
|
||||||
|
- If unset, cooldown is `1h`.
|
||||||
|
- If a rule fires outside cooldown:
|
||||||
|
- create incident;
|
||||||
|
- send configured alert channels;
|
||||||
|
- record dry-run remediation evidence if actions are configured.
|
||||||
|
- If a rule fires during cooldown:
|
||||||
|
- update rule last checked/matched/count fields;
|
||||||
|
- do not create a duplicate incident;
|
||||||
|
- do not send another alert;
|
||||||
|
- increment or record suppressed count if implemented.
|
||||||
|
|
||||||
|
## 6. Remediation/action model
|
||||||
|
|
||||||
|
Phase 1 should not execute destructive remediation. Implement guardrails and dry-run evidence.
|
||||||
|
|
||||||
|
Recommended Phase 1 action types:
|
||||||
|
|
||||||
|
- `generic_webhook`: dry-run by default; real execution can be deferred or restricted.
|
||||||
|
- `portainer_restart`: dry-run only in Phase 1.
|
||||||
|
- `allowlisted_command`: dry-run only in Phase 1; command IDs only, never arbitrary shell from UI/AI.
|
||||||
|
- `record_recommendation`: safe action that stores an advisory recommendation on the incident.
|
||||||
|
|
||||||
|
Guardrails:
|
||||||
|
|
||||||
|
- Global dry-run defaults true.
|
||||||
|
- Per-rule action must be enabled.
|
||||||
|
- Per-action dry-run should be explicit or inherited from global dry-run.
|
||||||
|
- AI may recommend actions but cannot execute them.
|
||||||
|
- No arbitrary command execution.
|
||||||
|
- No Portainer write API calls in Phase 1 unless separately approved.
|
||||||
|
|
||||||
|
Validation evidence:
|
||||||
|
|
||||||
|
- At least one rule with a configured dry-run action.
|
||||||
|
- Incident shows what would have happened.
|
||||||
|
- Logs do not expose secrets or raw full log payloads.
|
||||||
|
|
||||||
|
## 7. Local-only AI analysis
|
||||||
|
|
||||||
|
Local-only Ollama is approved for Phase 1. External/cloud AI providers are not approved.
|
||||||
|
|
||||||
|
Configuration:
|
||||||
|
|
||||||
|
- `LOG_GUARDIAN_ANALYSIS_PROVIDER=none|ollama`
|
||||||
|
- `LOG_GUARDIAN_OLLAMA_URL=http://ollama:11434`
|
||||||
|
- `LOG_GUARDIAN_OLLAMA_MODEL=<model>`
|
||||||
|
|
||||||
|
Default:
|
||||||
|
|
||||||
|
- `LOG_GUARDIAN_ANALYSIS_PROVIDER=none`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- AI analysis remains opt-in per rule.
|
||||||
|
- AI findings are advisory only.
|
||||||
|
- Limit number of log lines sent to Ollama.
|
||||||
|
- Redact obvious secrets before prompt construction.
|
||||||
|
- Do not log prompts or raw log payloads by default.
|
||||||
|
- Store only concise summaries/recommendations in incidents.
|
||||||
|
|
||||||
|
Prompt/redaction design must be documented before enabling Ollama in production.
|
||||||
|
|
||||||
|
## 8. UI scope
|
||||||
|
|
||||||
|
Add UI support for:
|
||||||
|
|
||||||
|
- Login-protected dashboard.
|
||||||
|
- Rule create/edit with cooldown and alert channel selection.
|
||||||
|
- Alert channel create/edit/test.
|
||||||
|
- Loki status/label discovery/query test.
|
||||||
|
- Incident list showing alert delivery state and remediation dry-run evidence where available.
|
||||||
|
|
||||||
|
Keep the UI served by Go. Do not add a server-side JavaScript runtime.
|
||||||
|
|
||||||
|
## 9. Tests and validation
|
||||||
|
|
||||||
|
Required implementation tests:
|
||||||
|
|
||||||
|
- Auth middleware unit tests.
|
||||||
|
- Store tests for alert channels, cooldown fields, and secret-preserving updates.
|
||||||
|
- Loki client tests using `httptest` fixtures for labels, label values, and query range.
|
||||||
|
- Alert dispatcher tests for ntfy using `httptest`.
|
||||||
|
- Rule engine tests for cooldown suppression.
|
||||||
|
- Remediation dry-run tests.
|
||||||
|
- Optional Ollama provider tests using `httptest` only; no live AI dependency in unit tests.
|
||||||
|
|
||||||
|
Required validation before Phase 1 completion:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
gofmt -w cmd internal
|
||||||
|
go test ./...
|
||||||
|
make validate-structure
|
||||||
|
make docker-build
|
||||||
|
```
|
||||||
|
|
||||||
|
Additional required evidence:
|
||||||
|
|
||||||
|
- Loki connectivity/query validation against the remote Loki server.
|
||||||
|
- Label discovery output documented without secrets.
|
||||||
|
- Self-hosted ntfy test alert received.
|
||||||
|
- A rule-generated ntfy alert received.
|
||||||
|
- Remediation dry-run evidence captured.
|
||||||
|
- Security/privacy review covering logs, prompts, secrets, alert tokens, and action permissions.
|
||||||
|
|
||||||
|
## 10. Required Phase 1 artifacts
|
||||||
|
|
||||||
|
Create/update these before declaring Phase 1 complete:
|
||||||
|
|
||||||
|
- `project-docs/status/phase-1-summary.md`
|
||||||
|
- `project-docs/status/phase-1-test-results.md`
|
||||||
|
- `project-docs/status/phase-1-open-issues.md`
|
||||||
|
|
||||||
|
## 11. Open planning items
|
||||||
|
|
||||||
|
- Final Loki topology: single-binary vs simple scalable.
|
||||||
|
- Final MinIO/NAS capacity and retention target after observing ingestion.
|
||||||
|
- Whether Gotify, generic webhook, or SMTP are implemented fully in Phase 1 or scaffolded after ntfy.
|
||||||
|
- Whether alert channel secrets are stored directly in local JSON or referenced via env/file paths.
|
||||||
|
- Exact Ollama model to use once Ollama is stable.
|
||||||
|
- Whether Log Guardian reaches Loki/ntfy/Ollama over LAN, VPN, or Docker overlay network.
|
||||||
29
project-docs/runbooks/loki-setup.md
Normal file
29
project-docs/runbooks/loki-setup.md
Normal file
@ -0,0 +1,29 @@
|
|||||||
|
# Loki Setup Runbook
|
||||||
|
|
||||||
|
## Starter local/Portainer stack
|
||||||
|
|
||||||
|
Use `deploy/loki/docker-compose.yml` and `deploy/loki/promtail-config.yml`.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd deploy/loki
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
## Verify Loki
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -fsS http://localhost:3100/ready
|
||||||
|
curl -G http://localhost:3100/loki/api/v1/labels
|
||||||
|
```
|
||||||
|
|
||||||
|
## Useful LogQL examples
|
||||||
|
|
||||||
|
```logql
|
||||||
|
{service="api"} |= "error"
|
||||||
|
{stack="second-brain"} |~ "(?i)(panic|fatal|exception)"
|
||||||
|
{stream="stderr"} |~ "(?i)(error|failed|timeout)"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Portainer notes
|
||||||
|
|
||||||
|
For Portainer deployment, mount the Docker socket and Docker container log directory read-only into Promtail. The included compose does this for a single Docker host. Multi-node Swarm setups need Promtail on each node or another log shipping strategy.
|
||||||
8
project-docs/status/phase-0-open-issues.md
Normal file
8
project-docs/status/phase-0-open-issues.md
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
# Phase 0 Open Issues
|
||||||
|
|
||||||
|
- Add UI authentication before exposing beyond a trusted private network.
|
||||||
|
- Validate the Loki + Promtail setup against actual Portainer/Docker hosts.
|
||||||
|
- Decide alert channels: email, ntfy, Gotify, Discord, Matrix, SMS, or another destination.
|
||||||
|
- Decide which remediation actions are allowed. Candidates: send webhook, restart Portainer service, scale service, pause updater, open issue, or run a tightly allowlisted command.
|
||||||
|
- AI analysis provider is intentionally unset. Any external AI/cloud provider requires explicit approval and prompt/log redaction design.
|
||||||
|
- Add real alert delivery, retry behavior, deduplication, silences, and escalation policies.
|
||||||
16
project-docs/status/phase-0-summary.md
Normal file
16
project-docs/status/phase-0-summary.md
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
# Phase 0 Summary
|
||||||
|
|
||||||
|
Created initial `log-guardian` scaffold.
|
||||||
|
|
||||||
|
Included:
|
||||||
|
|
||||||
|
- Go HTTP service with health endpoint and basic web UI.
|
||||||
|
- Local JSON-backed rule, alert-channel, and incident storage.
|
||||||
|
- Loki query client for LogQL range queries.
|
||||||
|
- Rule engine with thresholds and incident creation.
|
||||||
|
- No-op analyzer interface for future AI agent analysis.
|
||||||
|
- Dry-run remediation action runner scaffold.
|
||||||
|
- Alert dispatcher scaffold that logs alert intent only.
|
||||||
|
- Dockerfile, Portainer stack, Loki + Promtail starter compose, example environment, Makefile, CI workflow, and phase artifacts.
|
||||||
|
|
||||||
|
Phase 0 does not certify production alerting or remediation.
|
||||||
19
project-docs/status/phase-0-test-results.md
Normal file
19
project-docs/status/phase-0-test-results.md
Normal file
@ -0,0 +1,19 @@
|
|||||||
|
# Phase 0 Test Results
|
||||||
|
|
||||||
|
Validation run from `/data/code/log-guardian` after scaffold creation.
|
||||||
|
|
||||||
|
Passed:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
gofmt -w cmd internal
|
||||||
|
go test ./...
|
||||||
|
make validate-structure
|
||||||
|
make go-fmt
|
||||||
|
make docker-build
|
||||||
|
```
|
||||||
|
|
||||||
|
Evidence:
|
||||||
|
|
||||||
|
- `go test ./...` passed for all packages, including `internal/analysis` and `internal/store` tests.
|
||||||
|
- `make validate-structure` printed `Structure validation passed`.
|
||||||
|
- Docker image built successfully as `gitea.wayfinderak.com/wayfinderak/log-guardian:local`.
|
||||||
10
project-docs/status/phase-1-open-issues.md
Normal file
10
project-docs/status/phase-1-open-issues.md
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
# Phase 1 Open Issues
|
||||||
|
|
||||||
|
- Live deployment validation is still required against production Portainer, remote Loki, and self-hosted ntfy.
|
||||||
|
- Loki topology is not finalized: single-binary vs scalable deployment, exact MinIO/NAS capacity, and retention settings need confirmation.
|
||||||
|
- Promtail/Grafana Alloy labels must be discovered on the real Docker/Portainer hosts.
|
||||||
|
- Gotify, generic webhook, and SMTP channels are scaffolded but not implemented for real delivery.
|
||||||
|
- Alert channel secrets are currently stored in the local JSON config if entered through the UI; mounted secret-file/env references should be considered for stronger secret handling.
|
||||||
|
- Ollama provider is implemented as local-only and opt-in, but the final model and runtime deployment still need validation.
|
||||||
|
- No destructive remediation is implemented; Portainer restart and allowlisted command actions are dry-run evidence only.
|
||||||
|
- The UI supports create/replace workflows but does not yet provide polished edit forms for existing rules/channels.
|
||||||
16
project-docs/status/phase-1-summary.md
Normal file
16
project-docs/status/phase-1-summary.md
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
# Phase 1 Summary
|
||||||
|
|
||||||
|
Implemented Phase 1 foundations for production use:
|
||||||
|
|
||||||
|
- Basic auth protects UI/API routes; `/healthz` remains unauthenticated.
|
||||||
|
- Loki diagnostics endpoints were added for labels, label values, and limited query testing.
|
||||||
|
- Rules now support per-rule cooldowns with default `1h` and duplicate incident suppression.
|
||||||
|
- Alert channels are configurable in the UI.
|
||||||
|
- Real self-hosted ntfy delivery is implemented with test-send support.
|
||||||
|
- Gotify, generic webhook, and SMTP channel types are represented as scaffolded options.
|
||||||
|
- Incidents can record alert delivery results and remediation dry-run evidence.
|
||||||
|
- Remediation action runner supports guarded dry-run evidence for generic webhook, Portainer restart, allowlisted command, and recommendation actions.
|
||||||
|
- Local-only Ollama analysis support was added and remains disabled by default.
|
||||||
|
- Secret handling was improved for alert channel token preservation and display masking.
|
||||||
|
|
||||||
|
Phase 1 production rollout still requires live validation against the user's remote Loki and self-hosted ntfy deployments.
|
||||||
37
project-docs/status/phase-1-test-results.md
Normal file
37
project-docs/status/phase-1-test-results.md
Normal file
@ -0,0 +1,37 @@
|
|||||||
|
# Phase 1 Test Results
|
||||||
|
|
||||||
|
Validation run from `/data/code/log-guardian` after Phase 1 implementation.
|
||||||
|
|
||||||
|
Passed:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
gofmt -w cmd internal
|
||||||
|
go test ./...
|
||||||
|
make validate-structure
|
||||||
|
make go-fmt
|
||||||
|
make go-test
|
||||||
|
make docker-build
|
||||||
|
```
|
||||||
|
|
||||||
|
Docker image built successfully as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
gitea.wayfinderak.com/wayfinderak/log-guardian:local
|
||||||
|
```
|
||||||
|
|
||||||
|
Unit/integration-style tests added:
|
||||||
|
|
||||||
|
- Basic auth middleware protects UI routes and leaves `/healthz` unauthenticated.
|
||||||
|
- Store defaults include rule cooldown and alert channel secret-preserving updates.
|
||||||
|
- Loki client label, label-value, and query-range behavior uses `httptest` fixtures.
|
||||||
|
- ntfy dispatcher delivery uses an `httptest` fixture.
|
||||||
|
- Rule engine suppresses duplicate incidents during cooldown.
|
||||||
|
|
||||||
|
Pending live production validation before Phase 1 can be called operationally complete:
|
||||||
|
|
||||||
|
- Remote Loki connectivity and label discovery.
|
||||||
|
- Remote Loki LogQL query validation.
|
||||||
|
- Self-hosted ntfy test notification received.
|
||||||
|
- Rule-generated ntfy alert received.
|
||||||
|
- Remediation dry-run evidence captured from a real rule.
|
||||||
|
- Security/privacy review of logs, prompts, secrets, alert tokens, and action permissions.
|
||||||
1
web/static/app.css
Normal file
1
web/static/app.css
Normal file
@ -0,0 +1 @@
|
|||||||
|
:root{color-scheme:dark;--bg:#0d1117;--panel:#161b22;--text:#f0f6fc;--muted:#8b949e;--accent:#58a6ff;--danger:#ff7b72}*{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--text);font-family:system-ui,-apple-system,Segoe UI,sans-serif}.shell{max-width:1200px;margin:0 auto;padding:2rem}.card{background:var(--panel);border:1px solid #30363d;border-radius:16px;padding:1rem;margin:1rem 0}.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:1rem;align-items:end}.wide{grid-column:1/-1}label{display:grid;gap:.35rem;color:var(--muted)}.check{display:flex;gap:.5rem;align-items:center}input{width:100%;padding:.65rem;border-radius:10px;border:1px solid #30363d;background:#010409;color:var(--text)}button{padding:.65rem .9rem;border:0;border-radius:10px;background:var(--accent);color:#06111f;font-weight:700;cursor:pointer}.row{display:flex;justify-content:space-between;gap:1rem;align-items:center}table{width:100%;border-collapse:collapse}th,td{text-align:left;padding:.7rem;border-top:1px solid #30363d;vertical-align:top}code{color:#a5d6ff}.muted{color:var(--muted)}.error{color:var(--danger)}@media(max-width:760px){.shell{padding:1rem}table{display:block;overflow-x:auto}}
|
||||||
Loading…
x
Reference in New Issue
Block a user