Implement phase 1 foundations
All checks were successful
build-image / docker (push) Successful in 53s

This commit is contained in:
WayfinderAK 2026-07-03 20:14:06 -08:00
commit 6f7549d439
No known key found for this signature in database
39 changed files with 2221 additions and 0 deletions

5
.dockerignore Normal file
View File

@ -0,0 +1,5 @@
.git
.env
.env.*
data
.pi-subagents

15
.env.example Normal file
View File

@ -0,0 +1,15 @@
LOG_GUARDIAN_ADDR=:8080
LOG_GUARDIAN_DATA_PATH=./data/config.json
LOG_GUARDIAN_POLL_INTERVAL=1m
LOG_GUARDIAN_DRY_RUN_ACTIONS=true
LOG_GUARDIAN_AUTH_USERNAME=admin
LOG_GUARDIAN_AUTH_PASSWORD=change-me
LOKI_URL=http://localhost:3100
LOKI_TENANT_ID=
LOKI_USERNAME=
LOKI_PASSWORD=
LOG_GUARDIAN_ANALYSIS_PROVIDER=none
LOG_GUARDIAN_ANALYSIS_ENDPOINT=
LOG_GUARDIAN_ANALYSIS_TOKEN=
LOG_GUARDIAN_OLLAMA_URL=http://ollama:11434
LOG_GUARDIAN_OLLAMA_MODEL=llama3.1

View File

@ -0,0 +1,13 @@
name: build-image
on:
push:
branches: [main]
jobs:
docker:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build image
run: make docker-build IMAGE_TAG=${{ gitea.sha }}

17
.github/workflows/ci.yml vendored Normal file
View File

@ -0,0 +1,17 @@
name: ci
on:
push:
pull_request:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.23'
- run: make validate-structure
- run: make go-fmt
- run: make go-test

9
.gitignore vendored Normal file
View File

@ -0,0 +1,9 @@
.env
.env.*
!.env.example
data/
dist/
build/
coverage/
*.log
.DS_Store

33
AGENTS.md Normal file
View File

@ -0,0 +1,33 @@
# AGENTS.md
This repository is for a private log monitoring, alerting, and controlled remediation service built around Loki.
## Hard rules
- Server-side code must be Go.
- The web UI is served by Go; do not add a server-side JavaScript runtime.
- Never log alert destination secrets, API keys, tokens, AI prompts containing private logs, or full raw log payloads unless explicitly configured for a local-only debug session.
- Do not add external cloud/AI/data integrations without explicit user approval.
- Automatic remediation must be opt-in per rule and should default to dry-run.
- AI analysis is advisory by default; AI-generated actions must not execute unless a rule explicitly allows that action and passes guardrails.
- Preserve local data ownership: runtime configuration is stored in mounted local files unless a later phase explicitly changes storage.
## Required artifacts per phase
- `project-docs/status/phase-N-summary.md`
- `project-docs/status/phase-N-test-results.md`
- `project-docs/status/phase-N-open-issues.md`
## Required validation before advancing
Run the checks required by the phase, including as applicable:
- formatting/lint
- unit tests
- Docker build
- Loki query validation
- alert delivery validation
- remediation dry-run evidence
- security/privacy review for logs, prompts, secrets, and action permissions
If any required check fails, the phase is not complete.

31
Makefile Normal file
View File

@ -0,0 +1,31 @@
IMAGE_TAG ?= local
REGISTRY ?= gitea.wayfinderak.com
IMAGE_NAMESPACE ?= wayfinderak
IMAGE_PREFIX ?= log-guardian
IMAGE_REF ?= $(REGISTRY)/$(IMAGE_NAMESPACE)/$(IMAGE_PREFIX):$(IMAGE_TAG)
DOCKER_BUILD_FLAGS ?=
.PHONY: validate-structure go-fmt go-test docker-build ci-fast
validate-structure:
@test -f AGENTS.md
@test -f README.md
@test -f project-docs/status/phase-0-summary.md
@test -f project-docs/status/phase-0-test-results.md
@test -f project-docs/status/phase-0-open-issues.md
@test -f project-docs/status/phase-1-summary.md
@test -f project-docs/status/phase-1-test-results.md
@test -f project-docs/status/phase-1-open-issues.md
@test -f deploy/loki/docker-compose.yml
@echo "Structure validation passed"
go-fmt:
@test -z "$$(gofmt -l cmd internal)"
go-test:
go test ./...
docker-build:
docker build $(DOCKER_BUILD_FLAGS) -f docker/server.Dockerfile -t $(IMAGE_REF) .
ci-fast: validate-structure go-fmt go-test docker-build

92
README.md Normal file
View File

@ -0,0 +1,92 @@
# Log Guardian
Private Loki-backed service for log monitoring, alerting, and guarded remediation.
Current implementation includes the Phase 1 foundations:
- watches configured LogQL rules;
- records incidents when thresholds are met;
- enforces per-rule cooldowns;
- protects the UI with basic auth while leaving `/healthz` open;
- provides Loki label/query diagnostics;
- supports UI-configurable alert channels with real ntfy delivery;
- keeps remediation in dry-run by default;
- includes local-only Ollama analysis support, disabled by default.
## Run locally
```sh
cp .env.example .env
# edit env values, then export them or use your shell dotenv helper
go run ./cmd/server
```
Open <http://localhost:8080>.
## Configuration
| Variable | Default | Purpose |
| --- | --- | --- |
| `LOG_GUARDIAN_ADDR` | `:8080` | HTTP listen address. |
| `LOG_GUARDIAN_DATA_PATH` | `/data/config.json` | Local JSON config. |
| `LOG_GUARDIAN_POLL_INTERVAL` | `1m` | Rule evaluation interval. |
| `LOG_GUARDIAN_DRY_RUN_ACTIONS` | `true` | Prevents real remediation actions. |
| `LOG_GUARDIAN_AUTH_USERNAME` / `LOG_GUARDIAN_AUTH_PASSWORD` | empty | Required basic auth credentials for UI/API routes except `/healthz`. |
| `LOKI_URL` | `http://loki:3100` | Loki base URL. |
| `LOKI_TENANT_ID` | empty | Optional Loki tenant header. |
| `LOKI_USERNAME` / `LOKI_PASSWORD` | empty | Optional basic auth. |
| `LOG_GUARDIAN_ANALYSIS_PROVIDER` | `none` | `none` or local-only `ollama`. |
| `LOG_GUARDIAN_OLLAMA_URL` / `LOG_GUARDIAN_OLLAMA_MODEL` | `http://ollama:11434` / `llama3.1` | Local Ollama analysis config. |
## Initial Loki setup
A starter Loki + Promtail stack is in `deploy/loki/`:
```sh
cd deploy/loki
docker compose up -d
```
Promtail is configured to discover Docker containers and attach useful labels:
- `container`
- `service`
- `stack`
- `stream`
Example LogQL rules:
```logql
{service="api"} |= "error"
{stack="second-brain"} |~ "(?i)(panic|fatal|exception)"
{container=~".*postgres.*"} |= "database system is ready"
```
## Deploy in Portainer
Use `deploy/portainer-stack.yml` with values from `deploy/portainer.env.example`.
Keep this set while testing:
```text
LOG_GUARDIAN_DRY_RUN_ACTIONS=true
LOG_GUARDIAN_ANALYSIS_PROVIDER=none
```
## Build and validate
```sh
make ci-fast
```
## Safety model
Automatic remediation is intentionally constrained:
1. A rule must match.
2. The configured threshold must be exceeded.
3. The action must be explicitly enabled on that rule.
4. Global dry-run must be disabled.
5. Future AI recommendations are advisory unless a rule explicitly allows a matching action type.
Do not enable production remediation until a later phase explicitly approves concrete action types and guardrails.

49
cmd/server/main.go Normal file
View File

@ -0,0 +1,49 @@
package main
import (
"context"
"log"
"net/http"
"os/signal"
"syscall"
"time"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/actions"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/config"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/rules"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/server"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
func main() {
cfg := config.Load()
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
fileStore := store.NewFileStore(cfg.DataPath)
lokiClient := loki.New(cfg.LokiURL, cfg.LokiTenantID, cfg.LokiUsername, cfg.LokiPassword)
var analyzer analysis.Analyzer = analysis.NoopAnalyzer{}
if cfg.AnalysisProvider == "ollama" {
analyzer = analysis.NewOllamaAnalyzer(cfg.OllamaURL, cfg.OllamaModel)
}
actionRunner := actions.NewRunner(cfg.DryRunActions)
dispatcher := alerts.NewDispatcher()
engine := rules.NewEngine(fileStore, lokiClient, analyzer, actionRunner, dispatcher)
go engine.Start(ctx, cfg.PollInterval)
httpServer := &http.Server{Addr: cfg.Addr, Handler: server.New(fileStore, engine, lokiClient, dispatcher, cfg.AuthUsername, cfg.AuthPassword).Routes()}
go func() {
<-ctx.Done()
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = httpServer.Shutdown(shutdownCtx)
}()
log.Printf("log-guardian listening on %s", cfg.Addr)
if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatal(err)
}
}

View File

@ -0,0 +1,24 @@
services:
loki:
image: grafana/loki:3.2.1
command: -config.file=/etc/loki/local-config.yaml
ports:
- "${LOKI_PORT:-3100}:3100"
volumes:
- loki_data:/loki
restart: unless-stopped
promtail:
image: grafana/promtail:3.2.1
command: -config.file=/etc/promtail/config.yml
volumes:
- /var/log:/var/log:ro
- /var/lib/docker/containers:/var/lib/docker/containers:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./promtail-config.yml:/etc/promtail/config.yml:ro
depends_on:
- loki
restart: unless-stopped
volumes:
loki_data:

View File

@ -0,0 +1,25 @@
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://loki:3100/loki/api/v1/push
scrape_configs:
- job_name: docker
docker_sd_configs:
- host: unix:///var/run/docker.sock
refresh_interval: 10s
relabel_configs:
- source_labels: ['__meta_docker_container_name']
regex: '/(.*)'
target_label: 'container'
- source_labels: ['__meta_docker_container_label_com_docker_compose_service']
target_label: 'service'
- source_labels: ['__meta_docker_container_label_com_docker_compose_project']
target_label: 'stack'
- source_labels: ['__meta_docker_container_log_stream']
target_label: 'stream'

View File

@ -0,0 +1,28 @@
services:
log-guardian:
image: gitea.wayfinderak.com/wayfinderak/log-guardian:${IMAGE_TAG:-latest}
environment:
LOG_GUARDIAN_ADDR: :8080
LOG_GUARDIAN_DATA_PATH: /data/config.json
LOG_GUARDIAN_POLL_INTERVAL: ${LOG_GUARDIAN_POLL_INTERVAL:-1m}
LOG_GUARDIAN_DRY_RUN_ACTIONS: ${LOG_GUARDIAN_DRY_RUN_ACTIONS:-true}
LOKI_URL: ${LOKI_URL:-http://loki:3100}
LOKI_TENANT_ID: ${LOKI_TENANT_ID:-}
LOKI_USERNAME: ${LOKI_USERNAME:-}
LOKI_PASSWORD: ${LOKI_PASSWORD:-}
LOG_GUARDIAN_ANALYSIS_PROVIDER: ${LOG_GUARDIAN_ANALYSIS_PROVIDER:-none}
LOG_GUARDIAN_ANALYSIS_ENDPOINT: ${LOG_GUARDIAN_ANALYSIS_ENDPOINT:-}
LOG_GUARDIAN_ANALYSIS_TOKEN: ${LOG_GUARDIAN_ANALYSIS_TOKEN:-}
volumes:
- log_guardian_data:/data
ports:
- "${LOG_GUARDIAN_PORT:-8081}:8080"
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8080/healthz"]
interval: 30s
timeout: 5s
retries: 5
restart: unless-stopped
volumes:
log_guardian_data:

View File

@ -0,0 +1,11 @@
IMAGE_TAG=latest
LOG_GUARDIAN_PORT=8081
LOG_GUARDIAN_POLL_INTERVAL=1m
LOG_GUARDIAN_DRY_RUN_ACTIONS=true
LOKI_URL=http://loki:3100
LOKI_TENANT_ID=
LOKI_USERNAME=
LOKI_PASSWORD=
LOG_GUARDIAN_ANALYSIS_PROVIDER=none
LOG_GUARDIAN_ANALYSIS_ENDPOINT=
LOG_GUARDIAN_ANALYSIS_TOKEN=

19
docker/server.Dockerfile Normal file
View File

@ -0,0 +1,19 @@
FROM golang:1.23-alpine AS build
WORKDIR /src
COPY go.mod ./
COPY cmd ./cmd
COPY internal ./internal
COPY web ./web
RUN go build -trimpath -ldflags="-s -w" -o /out/log-guardian ./cmd/server
FROM alpine:3.20
LABEL org.opencontainers.image.title="log-guardian"
LABEL org.opencontainers.image.description="Loki-backed log monitoring, alerting, and guarded remediation service."
WORKDIR /app
RUN adduser -D -H guardian && mkdir -p /data && chown guardian:guardian /data
COPY --from=build /out/log-guardian /usr/local/bin/log-guardian
COPY --from=build /src/web ./web
USER guardian
EXPOSE 8080
VOLUME ["/data"]
CMD ["/usr/local/bin/log-guardian"]

3
go.mod Normal file
View File

@ -0,0 +1,3 @@
module gitea.wayfinderak.com/wayfinderak/log-guardian
go 1.23

View File

@ -0,0 +1,65 @@
package actions
import (
"context"
"fmt"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
type Result struct {
Action string `json:"action"`
DryRun bool `json:"dry_run"`
Detail string `json:"detail"`
}
type Runner struct {
dryRun bool
}
func NewRunner(dryRun bool) *Runner { return &Runner{dryRun: dryRun} }
func (r *Runner) Run(ctx context.Context, rule store.Rule, action store.Action) (Result, error) {
_ = ctx
if !action.Enabled {
return Result{Action: action.Type, DryRun: r.effectiveDryRun(action), Detail: "disabled"}, nil
}
switch action.Type {
case "generic_webhook", "webhook":
url := action.Params["url"]
if url == "" {
return Result{}, fmt.Errorf("webhook action requires url")
}
return Result{Action: action.Type, DryRun: true, Detail: "would POST webhook for rule " + rule.Name}, nil
case "record_recommendation":
return Result{Action: action.Type, DryRun: false, Detail: "recommendation recorded for operator review"}, nil
case "allowlisted_command", "command":
id := action.Params["command_id"]
if id == "" {
id = action.Params["id"]
}
if id == "" {
return Result{}, fmt.Errorf("allowlisted command requires command_id")
}
return Result{Action: action.Type, DryRun: true, Detail: "would run allowlisted command " + id}, nil
case "portainer_restart":
service := action.Params["service"]
if service == "" {
return Result{}, fmt.Errorf("portainer_restart requires service")
}
return Result{Action: action.Type, DryRun: true, Detail: "would restart Portainer service " + service}, nil
default:
return Result{}, fmt.Errorf("unsupported action type %q", action.Type)
}
return Result{}, fmt.Errorf("unsupported action type %q", action.Type)
}
func (r *Runner) effectiveDryRun(action store.Action) bool {
if r.dryRun {
return true
}
if action.DryRun == nil {
return true
}
return *action.DryRun
}

108
internal/alerts/alerts.go Normal file
View File

@ -0,0 +1,108 @@
package alerts
import (
"bytes"
"context"
"fmt"
"log"
"net/http"
"net/url"
"strings"
"time"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
type Result struct {
ChannelID string
Detail string
}
type Dispatcher struct {
http *http.Client
}
func NewDispatcher() *Dispatcher { return &Dispatcher{http: &http.Client{Timeout: 15 * time.Second}} }
func (d *Dispatcher) Send(ctx context.Context, rule store.Rule, finding analysis.Finding, channels []store.AlertChannel) ([]Result, error) {
if len(channels) == 0 {
log.Printf("alert for rule %q: %s", rule.Name, finding.Summary)
return []Result{{Detail: "no channels configured; logged alert intent"}}, nil
}
var results []Result
var firstErr error
for _, channel := range channels {
if !channel.Enabled {
results = append(results, Result{ChannelID: channel.ID, Detail: "disabled"})
continue
}
var err error
switch channel.Type {
case "ntfy":
err = d.sendNtfy(ctx, rule, finding, channel)
case "gotify", "generic_webhook", "smtp":
err = fmt.Errorf("%s delivery is scaffolded but not implemented in phase 1", channel.Type)
default:
err = fmt.Errorf("unsupported channel type %q", channel.Type)
}
if err != nil {
if firstErr == nil {
firstErr = err
}
results = append(results, Result{ChannelID: channel.ID, Detail: err.Error()})
continue
}
results = append(results, Result{ChannelID: channel.ID, Detail: "sent"})
}
return results, firstErr
}
func (d *Dispatcher) Test(ctx context.Context, channel store.AlertChannel) error {
finding := analysis.Finding{Summary: "Log Guardian test notification", Confidence: "test"}
rule := store.Rule{Name: "Test alert", Severity: "info"}
switch channel.Type {
case "ntfy":
return d.sendNtfy(ctx, rule, finding, channel)
case "gotify", "generic_webhook", "smtp":
return fmt.Errorf("%s delivery is scaffolded but not implemented in phase 1", channel.Type)
default:
return fmt.Errorf("unsupported channel type %q", channel.Type)
}
}
func (d *Dispatcher) sendNtfy(ctx context.Context, rule store.Rule, finding analysis.Finding, channel store.AlertChannel) error {
server := strings.TrimRight(channel.Params["server_url"], "/")
topic := strings.Trim(channel.Params["topic"], "/")
if server == "" || topic == "" {
return fmt.Errorf("ntfy requires server_url and topic")
}
target, err := url.JoinPath(server, topic)
if err != nil {
return err
}
body := rule.Name + ": " + finding.Summary
req, err := http.NewRequestWithContext(ctx, http.MethodPost, target, bytes.NewBufferString(body))
if err != nil {
return err
}
req.Header.Set("Title", "Log Guardian: "+rule.Severity)
if priority := channel.Params["priority"]; priority != "" {
req.Header.Set("Priority", priority)
}
if tags := channel.Params["tags"]; tags != "" {
req.Header.Set("Tags", tags)
}
if token := channel.Params["token"]; token != "" {
req.Header.Set("Authorization", "Bearer "+token)
}
resp, err := d.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode > 299 {
return fmt.Errorf("ntfy delivery failed: status %d", resp.StatusCode)
}
return nil
}

View File

@ -0,0 +1,31 @@
package alerts
import (
"net/http"
"net/http/httptest"
"testing"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
func TestNtfyDelivery(t *testing.T) {
var gotAuth string
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/alerts" {
t.Fatalf("unexpected path %s", r.URL.Path)
}
gotAuth = r.Header.Get("Authorization")
w.WriteHeader(http.StatusNoContent)
}))
defer ts.Close()
d := NewDispatcher()
_, err := d.Send(t.Context(), store.Rule{Name: "API", Severity: "warning"}, analysis.Finding{Summary: "failed"}, []store.AlertChannel{{ID: "c1", Name: "ntfy", Type: "ntfy", Enabled: true, Params: map[string]string{"server_url": ts.URL, "topic": "alerts", "token": "secret"}}})
if err != nil {
t.Fatal(err)
}
if gotAuth != "Bearer secret" {
t.Fatalf("missing auth header: %q", gotAuth)
}
}

View File

@ -0,0 +1,160 @@
package analysis
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
"unicode"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
type Finding struct {
Summary string `json:"summary"`
RecommendedActions []string `json:"recommended_actions,omitempty"`
Confidence string `json:"confidence"`
}
type Analyzer interface {
Analyze(ctx context.Context, rule store.Rule, matches []loki.Match) (Finding, error)
}
type NoopAnalyzer struct{}
func (NoopAnalyzer) Analyze(ctx context.Context, rule store.Rule, matches []loki.Match) (Finding, error) {
_ = ctx
return Finding{Summary: summarize(rule, matches), Confidence: "rule-only"}, nil
}
type OllamaAnalyzer struct {
URL string
Model string
http *http.Client
}
func NewOllamaAnalyzer(url, model string) *OllamaAnalyzer {
return &OllamaAnalyzer{URL: strings.TrimRight(url, "/"), Model: model, http: &http.Client{Timeout: 60 * time.Second}}
}
func (a *OllamaAnalyzer) Analyze(ctx context.Context, rule store.Rule, matches []loki.Match) (Finding, error) {
if a.URL == "" || a.Model == "" {
return NoopAnalyzer{}.Analyze(ctx, rule, matches)
}
prompt := buildPrompt(rule, matches)
body, _ := json.Marshal(map[string]any{"model": a.Model, "prompt": prompt, "stream": false})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, a.URL+"/api/generate", bytes.NewReader(body))
if err != nil {
return Finding{}, err
}
req.Header.Set("Content-Type", "application/json")
resp, err := a.http.Do(req)
if err != nil {
return Finding{}, err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode > 299 {
return Finding{}, fmt.Errorf("ollama analysis failed: status %d", resp.StatusCode)
}
var raw struct {
Response string `json:"response"`
}
if err := json.NewDecoder(resp.Body).Decode(&raw); err != nil {
return Finding{}, err
}
summary := strings.TrimSpace(raw.Response)
if summary == "" {
return NoopAnalyzer{}.Analyze(ctx, rule, matches)
}
if len(summary) > 500 {
summary = summary[:500] + "..."
}
return Finding{Summary: summary, Confidence: "ollama-local"}, nil
}
func buildPrompt(rule store.Rule, matches []loki.Match) string {
var b strings.Builder
b.WriteString("You are analyzing private application logs locally. Summarize likely cause and operator next steps in 4 concise sentences or fewer. Do not suggest destructive actions.\n")
b.WriteString("Rule: ")
b.WriteString(rule.Name)
b.WriteString("\nSeverity: ")
b.WriteString(rule.Severity)
b.WriteString("\nSanitized samples:\n")
limit := len(matches)
if limit > 5 {
limit = 5
}
for i := 0; i < limit; i++ {
line := sanitizeLine(matches[i].Line)
line = redactSecrets(line)
if len(line) > 300 {
line = line[:300] + "..."
}
b.WriteString("- ")
b.WriteString(line)
b.WriteByte('\n')
}
return b.String()
}
func summarize(rule store.Rule, matches []loki.Match) string {
if len(matches) == 0 {
return "No matching log entries."
}
line := redactSecrets(sanitizeLine(matches[0].Line))
if len(line) > 180 {
line = line[:180] + "..."
}
return rule.Name + " matched " + plural(len(matches), "log entry", "log entries") + "; first match: " + line
}
func sanitizeLine(line string) string {
line = strings.ReplaceAll(line, "\n", " ")
line = strings.ReplaceAll(line, "\r", " ")
return strings.TrimSpace(line)
}
func redactSecrets(line string) string {
fields := strings.Fields(line)
for i, field := range fields {
lower := strings.ToLower(field)
if strings.Contains(lower, "token=") || strings.Contains(lower, "password=") || strings.Contains(lower, "secret=") || strings.Contains(lower, "apikey=") || strings.Contains(lower, "api_key=") || strings.HasPrefix(lower, "bearer") {
fields[i] = redactField(field)
}
}
return strings.Join(fields, " ")
}
func redactField(field string) string {
idx := strings.IndexAny(field, "=:")
if idx < 0 {
return "[REDACTED]"
}
prefix := strings.TrimRightFunc(field[:idx], func(r rune) bool { return unicode.IsSpace(r) })
return prefix + "=[REDACTED]"
}
func plural(count int, singular, plural string) string {
if count == 1 {
return "1 " + singular
}
return strconvItoa(count) + " " + plural
}
func strconvItoa(i int) string {
if i == 0 {
return "0"
}
var b [20]byte
bp := len(b)
for i > 0 {
bp--
b[bp] = byte('0' + i%10)
i /= 10
}
return string(b[bp:])
}

View File

@ -0,0 +1,10 @@
package analysis
import "testing"
func TestSanitizeLine(t *testing.T) {
got := sanitizeLine(" error\nwith\rspaces ")
if got != "error with spaces" {
t.Fatalf("got %q", got)
}
}

83
internal/config/config.go Normal file
View File

@ -0,0 +1,83 @@
package config
import (
"os"
"strconv"
"time"
)
type Config struct {
Addr string
DataPath string
PollInterval time.Duration
LokiURL string
LokiTenantID string
LokiUsername string
LokiPassword string
DryRunActions bool
AnalysisProvider string
AnalysisEndpoint string
AnalysisToken string
OllamaURL string
OllamaModel string
AuthUsername string
AuthPassword string
}
func Load() Config {
return Config{
Addr: env("LOG_GUARDIAN_ADDR", ":8080"),
DataPath: env("LOG_GUARDIAN_DATA_PATH", "/data/config.json"),
PollInterval: durationEnv("LOG_GUARDIAN_POLL_INTERVAL", 1*time.Minute),
LokiURL: trimRightSlash(env("LOKI_URL", "http://loki:3100")),
LokiTenantID: env("LOKI_TENANT_ID", ""),
LokiUsername: env("LOKI_USERNAME", ""),
LokiPassword: env("LOKI_PASSWORD", ""),
DryRunActions: boolEnv("LOG_GUARDIAN_DRY_RUN_ACTIONS", true),
AnalysisProvider: env("LOG_GUARDIAN_ANALYSIS_PROVIDER", "none"),
AnalysisEndpoint: env("LOG_GUARDIAN_ANALYSIS_ENDPOINT", ""),
AnalysisToken: env("LOG_GUARDIAN_ANALYSIS_TOKEN", ""),
OllamaURL: trimRightSlash(env("LOG_GUARDIAN_OLLAMA_URL", "http://ollama:11434")),
OllamaModel: env("LOG_GUARDIAN_OLLAMA_MODEL", "llama3.1"),
AuthUsername: env("LOG_GUARDIAN_AUTH_USERNAME", ""),
AuthPassword: env("LOG_GUARDIAN_AUTH_PASSWORD", ""),
}
}
func env(key, fallback string) string {
if value := os.Getenv(key); value != "" {
return value
}
return fallback
}
func boolEnv(key string, fallback bool) bool {
value := os.Getenv(key)
if value == "" {
return fallback
}
parsed, err := strconv.ParseBool(value)
if err != nil {
return fallback
}
return parsed
}
func durationEnv(key string, fallback time.Duration) time.Duration {
value := os.Getenv(key)
if value == "" {
return fallback
}
parsed, err := time.ParseDuration(value)
if err != nil {
return fallback
}
return parsed
}
func trimRightSlash(value string) string {
for len(value) > 1 && value[len(value)-1] == '/' {
value = value[:len(value)-1]
}
return value
}

148
internal/loki/client.go Normal file
View File

@ -0,0 +1,148 @@
package loki
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
type Client struct {
baseURL string
tenantID string
username string
password string
http *http.Client
}
func New(baseURL, tenantID, username, password string) *Client {
return &Client{baseURL: strings.TrimRight(baseURL, "/"), tenantID: tenantID, username: username, password: password, http: &http.Client{Timeout: 20 * time.Second}}
}
type Match struct {
Timestamp time.Time `json:"timestamp"`
Line string `json:"line"`
}
type QueryResult struct {
Count int `json:"count"`
Matches []Match `json:"matches"`
}
func (c *Client) Ready(ctx context.Context) error {
req, err := c.request(ctx, http.MethodGet, "/ready", nil)
if err != nil {
return err
}
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode > 299 {
return fmt.Errorf("loki ready failed: status %d", resp.StatusCode)
}
return nil
}
func (c *Client) Labels(ctx context.Context) ([]string, error) {
var raw labelsResponse
if err := c.getJSON(ctx, "/loki/api/v1/labels", nil, &raw); err != nil {
return nil, err
}
return raw.Data, nil
}
func (c *Client) LabelValues(ctx context.Context, name string) ([]string, error) {
if name == "" || strings.Contains(name, "/") {
return nil, fmt.Errorf("invalid label name")
}
var raw labelsResponse
if err := c.getJSON(ctx, "/loki/api/v1/label/"+url.PathEscape(name)+"/values", nil, &raw); err != nil {
return nil, err
}
return raw.Data, nil
}
func (c *Client) QueryRange(ctx context.Context, logql string, since time.Duration, limit int) (QueryResult, error) {
if limit <= 0 {
limit = 100
}
values := url.Values{}
values.Set("query", logql)
values.Set("start", strconv.FormatInt(time.Now().Add(-since).UnixNano(), 10))
values.Set("end", strconv.FormatInt(time.Now().UnixNano(), 10))
values.Set("limit", strconv.Itoa(limit))
var raw response
if err := c.getJSON(ctx, "/loki/api/v1/query_range", values, &raw); err != nil {
return QueryResult{}, err
}
var result QueryResult
for _, stream := range raw.Data.Result {
for _, pair := range stream.Values {
if len(pair) != 2 {
continue
}
ns, _ := strconv.ParseInt(pair[0], 10, 64)
result.Matches = append(result.Matches, Match{Timestamp: time.Unix(0, ns), Line: pair[1]})
}
}
result.Count = len(result.Matches)
return result, nil
}
func (c *Client) getJSON(ctx context.Context, path string, values url.Values, out any) error {
req, err := c.request(ctx, http.MethodGet, path, values)
if err != nil {
return err
}
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode > 299 {
return fmt.Errorf("loki request failed: status %d", resp.StatusCode)
}
return json.NewDecoder(resp.Body).Decode(out)
}
func (c *Client) request(ctx context.Context, method, path string, values url.Values) (*http.Request, error) {
if c.baseURL == "" {
return nil, fmt.Errorf("LOKI_URL is not configured")
}
target := c.baseURL + path
if len(values) > 0 {
target += "?" + values.Encode()
}
req, err := http.NewRequestWithContext(ctx, method, target, nil)
if err != nil {
return nil, err
}
if c.tenantID != "" {
req.Header.Set("X-Scope-OrgID", c.tenantID)
}
if c.username != "" || c.password != "" {
req.SetBasicAuth(c.username, c.password)
}
return req, nil
}
type labelsResponse struct {
Status string `json:"status"`
Data []string `json:"data"`
}
type response struct {
Status string `json:"status"`
Data struct {
Result []struct {
Stream map[string]string `json:"stream"`
Values [][]string `json:"values"`
} `json:"result"`
} `json:"data"`
}

View File

@ -0,0 +1,42 @@
package loki
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
func TestLabelsValuesAndQueryRange(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/loki/api/v1/labels":
_, _ = w.Write([]byte(`{"status":"success","data":["service","container"]}`))
case "/loki/api/v1/label/service/values":
_, _ = w.Write([]byte(`{"status":"success","data":["api"]}`))
case "/loki/api/v1/query_range":
if !strings.Contains(r.URL.Query().Get("query"), "api") {
t.Fatalf("query not passed through: %s", r.URL.RawQuery)
}
_, _ = w.Write([]byte(`{"status":"success","data":{"result":[{"stream":{"service":"api"},"values":[["1700000000000000000","error token=secret"]]}]}}`))
default:
http.NotFound(w, r)
}
}))
defer ts.Close()
client := New(ts.URL, "", "", "")
labels, err := client.Labels(t.Context())
if err != nil || len(labels) != 2 {
t.Fatalf("labels=%v err=%v", labels, err)
}
values, err := client.LabelValues(t.Context(), "service")
if err != nil || values[0] != "api" {
t.Fatalf("values=%v err=%v", values, err)
}
result, err := client.QueryRange(t.Context(), `{service="api"}`, time.Minute, 10)
if err != nil || result.Count != 1 {
t.Fatalf("result=%#v err=%v", result, err)
}
}

152
internal/rules/engine.go Normal file
View File

@ -0,0 +1,152 @@
package rules
import (
"context"
"log"
"time"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/actions"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
type Store interface {
Load() (store.Data, error)
UpdateRule(id string, fn func(*store.Rule)) error
AddIncident(incident store.Incident) error
}
type Engine struct {
store Store
loki *loki.Client
analyzer analysis.Analyzer
actions *actions.Runner
dispatcher *alerts.Dispatcher
}
func NewEngine(s Store, lokiClient *loki.Client, analyzer analysis.Analyzer, actionRunner *actions.Runner, dispatcher *alerts.Dispatcher) *Engine {
return &Engine{store: s, loki: lokiClient, analyzer: analyzer, actions: actionRunner, dispatcher: dispatcher}
}
func (e *Engine) Start(ctx context.Context, interval time.Duration) {
if interval <= 0 {
interval = time.Minute
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
e.CheckAll(ctx)
select {
case <-ctx.Done():
return
case <-ticker.C:
}
}
}
func (e *Engine) CheckAll(ctx context.Context) {
data, err := e.store.Load()
if err != nil {
log.Printf("load rules failed: %v", err)
return
}
channels := map[string]store.AlertChannel{}
for _, channel := range data.AlertChannels {
channels[channel.ID] = channel
}
for _, rule := range data.Rules {
if !rule.Enabled {
continue
}
if err := e.checkRule(ctx, rule, channels); err != nil {
log.Printf("rule %q check failed: %v", rule.Name, err)
}
}
}
func (e *Engine) checkRule(ctx context.Context, rule store.Rule, channels map[string]store.AlertChannel) error {
window, err := time.ParseDuration(rule.Window)
if err != nil {
window = 5 * time.Minute
}
result, err := e.loki.QueryRange(ctx, rule.LogQL, window, 200)
if err != nil {
e.record(rule.ID, 0, err)
return err
}
if result.Count < rule.Threshold {
return e.record(rule.ID, result.Count, nil)
}
now := time.Now().UTC()
cooldown, err := time.ParseDuration(rule.Cooldown)
if err != nil || cooldown <= 0 {
cooldown = time.Hour
}
if !rule.LastAlertedAt.IsZero() && now.Sub(rule.LastAlertedAt) < cooldown {
return e.store.UpdateRule(rule.ID, func(r *store.Rule) {
r.LastCheckedAt = now
r.LastMatchedAt = now
r.LastMatchCount = result.Count
r.SuppressedCount++
r.LastError = ""
})
}
finding, err := e.analyzer.Analyze(ctx, rule, result.Matches)
if err != nil {
e.record(rule.ID, result.Count, err)
return err
}
selected := make([]store.AlertChannel, 0, len(rule.AlertChannels))
for _, id := range rule.AlertChannels {
if channel, ok := channels[id]; ok {
selected = append(selected, channel)
}
}
alertResults, alertErr := e.dispatcher.Send(ctx, rule, finding, selected)
var alertEvidence []string
for _, result := range alertResults {
if result.ChannelID == "" {
alertEvidence = append(alertEvidence, result.Detail)
} else {
alertEvidence = append(alertEvidence, result.ChannelID+": "+result.Detail)
}
}
var actionEvidence []string
for _, action := range rule.Actions {
result, err := e.actions.Run(ctx, rule, action)
if err != nil {
log.Printf("action %q for rule %q failed: %v", action.Type, rule.Name, err)
actionEvidence = append(actionEvidence, action.Type+": "+err.Error())
continue
}
actionEvidence = append(actionEvidence, result.Action+": "+result.Detail)
}
incident := store.Incident{RuleID: rule.ID, RuleName: rule.Name, Severity: rule.Severity, Count: result.Count, Summary: finding.Summary, AlertResults: alertEvidence, RemediationEvidence: actionEvidence, CreatedAt: now}
if err := e.store.AddIncident(incident); err != nil {
return err
}
if err := e.store.UpdateRule(rule.ID, func(r *store.Rule) {
r.LastCheckedAt = now
r.LastMatchedAt = now
r.LastAlertedAt = now
r.LastMatchCount = result.Count
r.LastError = ""
}); err != nil {
return err
}
return alertErr
}
func (e *Engine) record(id string, count int, err error) error {
return e.store.UpdateRule(id, func(r *store.Rule) {
r.LastCheckedAt = time.Now().UTC()
r.LastMatchCount = count
if err != nil {
r.LastError = err.Error()
} else {
r.LastError = ""
}
})
}

View File

@ -0,0 +1,42 @@
package rules
import (
"net/http"
"net/http/httptest"
"testing"
"time"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/actions"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/analysis"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
func TestCooldownSuppressesDuplicateIncident(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"status":"success","data":{"result":[{"stream":{},"values":[["1700000000000000000","error"]]}]}}`))
}))
defer ts.Close()
s := store.NewFileStore(t.TempDir() + "/config.json")
if err := s.UpsertRule(store.Rule{Name: "Errors", Enabled: true, LogQL: `{service="api"}`, Threshold: 1, Window: "5m", Cooldown: "1h", Actions: []store.Action{{Type: "record_recommendation", Enabled: true}}}); err != nil {
t.Fatal(err)
}
engine := NewEngine(s, loki.New(ts.URL, "", "", ""), analysis.NoopAnalyzer{}, actions.NewRunner(true), alerts.NewDispatcher())
engine.CheckAll(t.Context())
engine.CheckAll(t.Context())
data, err := s.Load()
if err != nil {
t.Fatal(err)
}
if len(data.Incidents) != 1 {
t.Fatalf("expected one incident, got %d", len(data.Incidents))
}
if data.Rules[0].SuppressedCount != 1 {
t.Fatalf("expected suppressed count 1, got %d", data.Rules[0].SuppressedCount)
}
if time.Since(data.Rules[0].LastAlertedAt) > time.Minute {
t.Fatalf("last alerted not set: %s", data.Rules[0].LastAlertedAt)
}
}

View File

@ -0,0 +1,51 @@
package server
import (
"crypto/subtle"
"encoding/json"
"log"
"net/http"
"time"
)
func writeJSON(w http.ResponseWriter, v any) {
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(v)
}
func requestLog(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
next.ServeHTTP(w, r)
log.Printf("%s %s %s", r.Method, r.URL.Path, time.Since(start).Round(time.Millisecond))
})
}
func securityHeaders(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("Referrer-Policy", "no-referrer")
w.Header().Set("Content-Security-Policy", "default-src 'self'; style-src 'self' 'unsafe-inline'; form-action 'self'")
next.ServeHTTP(w, r)
})
}
func basicAuth(next http.Handler, username, password string) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/healthz" {
next.ServeHTTP(w, r)
return
}
if username == "" || password == "" {
http.Error(w, "basic auth is not configured", http.StatusServiceUnavailable)
return
}
gotUser, gotPass, ok := r.BasicAuth()
if !ok || subtle.ConstantTimeCompare([]byte(gotUser), []byte(username)) != 1 || subtle.ConstantTimeCompare([]byte(gotPass), []byte(password)) != 1 {
w.Header().Set("WWW-Authenticate", `Basic realm="log-guardian"`)
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
next.ServeHTTP(w, r)
})
}

View File

@ -0,0 +1,34 @@
package server
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestBasicAuthAllowsHealthzWithoutCredentials(t *testing.T) {
h := basicAuth(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) }), "admin", "secret")
req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
if rr.Code != http.StatusNoContent {
t.Fatalf("got %d", rr.Code)
}
}
func TestBasicAuthProtectsUI(t *testing.T) {
h := basicAuth(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusNoContent) }), "admin", "secret")
req := httptest.NewRequest(http.MethodGet, "/", nil)
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
if rr.Code != http.StatusUnauthorized {
t.Fatalf("got %d", rr.Code)
}
req = httptest.NewRequest(http.MethodGet, "/", nil)
req.SetBasicAuth("admin", "secret")
rr = httptest.NewRecorder()
h.ServeHTTP(rr, req)
if rr.Code != http.StatusNoContent {
t.Fatalf("got %d", rr.Code)
}
}

202
internal/server/server.go Normal file
View File

@ -0,0 +1,202 @@
package server
import (
"context"
"html/template"
"net/http"
"strconv"
"strings"
"time"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/alerts"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/loki"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/rules"
"gitea.wayfinderak.com/wayfinderak/log-guardian/internal/store"
)
type Server struct {
store *store.FileStore
engine *rules.Engine
loki *loki.Client
dispatcher *alerts.Dispatcher
authUsername string
authPassword string
}
func New(s *store.FileStore, engine *rules.Engine, lokiClient *loki.Client, dispatcher *alerts.Dispatcher, authUsername, authPassword string) *Server {
return &Server{store: s, engine: engine, loki: lokiClient, dispatcher: dispatcher, authUsername: authUsername, authPassword: authPassword}
}
func (s *Server) Routes() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("GET /healthz", s.health)
mux.HandleFunc("GET /", s.index)
mux.HandleFunc("POST /rules", s.saveRule)
mux.HandleFunc("POST /check", s.checkNow)
mux.HandleFunc("POST /channels", s.saveChannel)
mux.HandleFunc("POST /channels/delete", s.deleteChannel)
mux.HandleFunc("POST /channels/test", s.testChannel)
mux.HandleFunc("GET /loki/labels", s.lokiLabels)
mux.HandleFunc("GET /loki/label-values", s.lokiLabelValues)
mux.HandleFunc("POST /loki/query-test", s.lokiQueryTest)
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServer(http.Dir("web/static"))))
return securityHeaders(requestLog(basicAuth(mux, s.authUsername, s.authPassword)))
}
func (s *Server) health(w http.ResponseWriter, r *http.Request) {
writeJSON(w, map[string]string{"status": "ok", "service": "log-guardian"})
}
func (s *Server) index(w http.ResponseWriter, r *http.Request) {
data, err := s.store.Load()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
render(w, data)
}
func (s *Server) saveRule(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
threshold, _ := strconv.Atoi(r.FormValue("threshold"))
rule := store.Rule{ID: strings.TrimSpace(r.FormValue("id")), Name: strings.TrimSpace(r.FormValue("name")), Enabled: r.FormValue("enabled") == "on", LogQL: strings.TrimSpace(r.FormValue("logql")), Severity: strings.TrimSpace(r.FormValue("severity")), Threshold: threshold, Window: strings.TrimSpace(r.FormValue("window")), Cooldown: strings.TrimSpace(r.FormValue("cooldown")), AnalysisEnabled: r.FormValue("analysis_enabled") == "on", AlertChannels: r.Form["alert_channels"]}
if rule.Name == "" || rule.LogQL == "" {
http.Error(w, "name and LogQL are required", http.StatusBadRequest)
return
}
if rule.Severity == "" {
rule.Severity = "warning"
}
if err := s.store.UpsertRule(rule); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
http.Redirect(w, r, "/", http.StatusSeeOther)
}
func (s *Server) saveChannel(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
channel := store.AlertChannel{ID: strings.TrimSpace(r.FormValue("id")), Name: strings.TrimSpace(r.FormValue("name")), Type: strings.TrimSpace(r.FormValue("type")), Enabled: r.FormValue("enabled") == "on", Params: map[string]string{}}
for _, key := range []string{"server_url", "topic", "token", "priority", "tags", "url", "api_url", "email"} {
if value := strings.TrimSpace(r.FormValue(key)); value != "" {
channel.Params[key] = value
}
}
if channel.Name == "" || channel.Type == "" {
http.Error(w, "name and type are required", http.StatusBadRequest)
return
}
if err := s.store.UpsertAlertChannel(channel); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
http.Redirect(w, r, "/", http.StatusSeeOther)
}
func (s *Server) deleteChannel(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if err := s.store.DeleteAlertChannel(r.FormValue("id")); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
http.Redirect(w, r, "/", http.StatusSeeOther)
}
func (s *Server) testChannel(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
data, err := s.store.Load()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
for _, channel := range data.AlertChannels {
if channel.ID == r.FormValue("id") {
ctx, cancel := context.WithTimeout(r.Context(), 15*time.Second)
defer cancel()
if err := s.dispatcher.Test(ctx, channel); err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
writeJSON(w, map[string]string{"status": "sent"})
return
}
}
http.Error(w, "channel not found", http.StatusNotFound)
}
func (s *Server) checkNow(w http.ResponseWriter, r *http.Request) {
s.engine.CheckAll(context.Background())
http.Redirect(w, r, "/", http.StatusSeeOther)
}
func (s *Server) lokiLabels(w http.ResponseWriter, r *http.Request) {
labels, err := s.loki.Labels(r.Context())
if err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
writeJSON(w, map[string]any{"labels": labels})
}
func (s *Server) lokiLabelValues(w http.ResponseWriter, r *http.Request) {
values, err := s.loki.LabelValues(r.Context(), r.URL.Query().Get("name"))
if err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
writeJSON(w, map[string]any{"values": values})
}
func (s *Server) lokiQueryTest(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
window, err := time.ParseDuration(r.FormValue("window"))
if err != nil {
window = 5 * time.Minute
}
result, err := s.loki.QueryRange(r.Context(), strings.TrimSpace(r.FormValue("logql")), window, 20)
if err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
if len(result.Matches) > 5 {
result.Matches = result.Matches[:5]
}
writeJSON(w, result)
}
var tmpl = template.Must(template.New("index").Funcs(template.FuncMap{"secretState": secretState}).Parse(`<!doctype html>
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>Log Guardian</title><link rel="stylesheet" href="/static/app.css"></head>
<body><main class="shell"><header><h1>Log Guardian</h1><p>Loki-backed log monitoring, alerting, and guarded remediation.</p></header>
<section class="card"><h2>Loki diagnostics</h2><p>Use authenticated endpoints: <code>/loki/labels</code>, <code>/loki/label-values?name=service</code>.</p><form method="post" action="/loki/query-test" class="grid"><label class="wide">Test LogQL <input name="logql" placeholder='{service="api"} |= "error"'></label><label>Window <input name="window" value="5m"></label><button type="submit">Return JSON sample</button></form></section>
<section class="card"><h2>Add or replace rule</h2><form method="post" action="/rules" class="grid"><label>Name <input name="name" required placeholder="API errors"></label><label>ID <input name="id" placeholder="optional stable id"></label><label>Severity <input name="severity" value="warning"></label><label>Window <input name="window" value="5m"></label><label>Cooldown <input name="cooldown" value="1h"></label><label>Threshold <input name="threshold" type="number" min="1" value="1"></label><label class="wide">LogQL <input name="logql" required placeholder='{service="api"} |= "error"'></label><fieldset class="wide"><legend>Alert channels</legend>{{range .AlertChannels}}<label class="check"><input type="checkbox" name="alert_channels" value="{{.ID}}"> {{.Name}} ({{.Type}})</label>{{else}}<span class="muted">No channels configured.</span>{{end}}</fieldset><label class="check"><input type="checkbox" name="enabled" checked> Enabled</label><label class="check"><input type="checkbox" name="analysis_enabled"> AI analysis</label><button type="submit">Save rule</button></form></section>
<section class="card"><h2>Alert channels</h2><form method="post" action="/channels" class="grid"><label>Name <input name="name" required></label><label>ID <input name="id" placeholder="optional stable id"></label><label>Type <select name="type"><option value="ntfy">ntfy</option><option value="gotify">gotify (scaffold)</option><option value="generic_webhook">generic webhook (scaffold)</option><option value="smtp">smtp (scaffold)</option></select></label><label class="check"><input type="checkbox" name="enabled" checked> Enabled</label><label>Server URL <input name="server_url" placeholder="https://ntfy.example.com"></label><label>Topic <input name="topic"></label><label>Token <input name="token" type="password" placeholder="leave blank to preserve"></label><label>Priority <input name="priority" placeholder="default, high, urgent"></label><label>Tags <input name="tags" placeholder="warning,rotating_light"></label><button type="submit">Save channel</button></form><table><thead><tr><th>Name</th><th>Type</th><th>Status</th><th>Secret</th><th>Actions</th></tr></thead><tbody>{{range .AlertChannels}}<tr><td>{{.Name}}</td><td>{{.Type}}</td><td>{{if .Enabled}}enabled{{else}}disabled{{end}}</td><td>{{secretState .Params}}</td><td><form method="post" action="/channels/test" class="inline"><input type="hidden" name="id" value="{{.ID}}"><button>Test</button></form><form method="post" action="/channels/delete" class="inline"><input type="hidden" name="id" value="{{.ID}}"><button>Delete</button></form></td></tr>{{else}}<tr><td colspan="5" class="muted">No alert channels configured.</td></tr>{{end}}</tbody></table></section>
<section class="card"><div class="row"><h2>Rules</h2><form method="post" action="/check"><button type="submit">Check now</button></form></div><table><thead><tr><th>Name</th><th>LogQL</th><th>Cooldown</th><th>Last count</th><th>Suppressed</th><th>Status</th></tr></thead><tbody>{{range .Rules}}<tr><td>{{.Name}}{{if not .Enabled}} <span class="muted">disabled</span>{{end}}</td><td><code>{{.LogQL}}</code></td><td>{{.Cooldown}}</td><td>{{.LastMatchCount}}</td><td>{{.SuppressedCount}}</td><td>{{if .LastError}}<span class="error">{{.LastError}}</span>{{else}}checked {{.LastCheckedAt}}{{end}}</td></tr>{{else}}<tr><td colspan="6" class="muted">No rules configured yet.</td></tr>{{end}}</tbody></table></section>
<section class="card"><h2>Recent incidents</h2><table><thead><tr><th>Time</th><th>Rule</th><th>Severity</th><th>Summary</th><th>Evidence</th></tr></thead><tbody>{{range .Incidents}}<tr><td>{{.CreatedAt}}</td><td>{{.RuleName}}</td><td>{{.Severity}}</td><td>{{.Summary}}</td><td>{{range .AlertResults}}<div>{{.}}</div>{{end}}{{range .RemediationEvidence}}<div>{{.}}</div>{{end}}</td></tr>{{else}}<tr><td colspan="5" class="muted">No incidents recorded.</td></tr>{{end}}</tbody></table></section>
</main></body></html>`))
func secretState(params map[string]string) string {
if params["token"] != "" || params["password"] != "" || params["secret"] != "" {
return "configured"
}
return "not configured"
}
func render(w http.ResponseWriter, data store.Data) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_ = tmpl.Execute(w, data)
}

260
internal/store/store.go Normal file
View File

@ -0,0 +1,260 @@
package store
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
type Rule struct {
ID string `json:"id"`
Name string `json:"name"`
Enabled bool `json:"enabled"`
LogQL string `json:"logql"`
Severity string `json:"severity"`
Threshold int `json:"threshold"`
Window string `json:"window"`
Cooldown string `json:"cooldown,omitempty"`
AlertChannels []string `json:"alert_channels,omitempty"`
AnalysisEnabled bool `json:"analysis_enabled"`
Actions []Action `json:"actions,omitempty"`
LastCheckedAt time.Time `json:"last_checked_at,omitempty"`
LastMatchedAt time.Time `json:"last_matched_at,omitempty"`
LastAlertedAt time.Time `json:"last_alerted_at,omitempty"`
LastMatchCount int `json:"last_match_count,omitempty"`
SuppressedCount int `json:"suppressed_count,omitempty"`
LastError string `json:"last_error,omitempty"`
}
type Action struct {
Type string `json:"type"`
Enabled bool `json:"enabled"`
DryRun *bool `json:"dry_run,omitempty"`
Params map[string]string `json:"params,omitempty"`
}
type AlertChannel struct {
ID string `json:"id"`
Name string `json:"name"`
Type string `json:"type"`
Enabled bool `json:"enabled"`
Params map[string]string `json:"params,omitempty"`
}
type Incident struct {
ID string `json:"id"`
RuleID string `json:"rule_id"`
RuleName string `json:"rule_name"`
Severity string `json:"severity"`
Count int `json:"count"`
Summary string `json:"summary"`
AlertResults []string `json:"alert_results,omitempty"`
RemediationEvidence []string `json:"remediation_evidence,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
type Data struct {
Rules []Rule `json:"rules"`
AlertChannels []AlertChannel `json:"alert_channels"`
Incidents []Incident `json:"incidents"`
}
type FileStore struct {
path string
mu sync.Mutex
}
func NewFileStore(path string) *FileStore { return &FileStore{path: path} }
func (s *FileStore) Load() (Data, error) {
s.mu.Lock()
defer s.mu.Unlock()
return s.loadLocked()
}
func (s *FileStore) Save(data Data) error {
s.mu.Lock()
defer s.mu.Unlock()
return s.saveLocked(data)
}
func (s *FileStore) UpsertRule(rule Rule) error {
s.mu.Lock()
defer s.mu.Unlock()
data, err := s.loadLocked()
if err != nil {
return err
}
if rule.ID == "" {
rule.ID = slug(rule.Name)
}
if rule.ID == "" {
return errors.New("rule id or name is required")
}
if rule.Threshold <= 0 {
rule.Threshold = 1
}
if rule.Window == "" {
rule.Window = "5m"
}
if rule.Cooldown == "" {
rule.Cooldown = "1h"
}
for i := range data.Rules {
if data.Rules[i].ID == rule.ID {
rule.LastCheckedAt = data.Rules[i].LastCheckedAt
rule.LastMatchedAt = data.Rules[i].LastMatchedAt
rule.LastAlertedAt = data.Rules[i].LastAlertedAt
rule.LastMatchCount = data.Rules[i].LastMatchCount
rule.SuppressedCount = data.Rules[i].SuppressedCount
rule.LastError = data.Rules[i].LastError
data.Rules[i] = rule
return s.saveLocked(data)
}
}
data.Rules = append(data.Rules, rule)
return s.saveLocked(data)
}
func (s *FileStore) UpsertAlertChannel(channel AlertChannel) error {
s.mu.Lock()
defer s.mu.Unlock()
data, err := s.loadLocked()
if err != nil {
return err
}
if channel.ID == "" {
channel.ID = slug(channel.Name)
}
if channel.ID == "" {
return errors.New("channel id or name is required")
}
if channel.Params == nil {
channel.Params = map[string]string{}
}
for i := range data.AlertChannels {
if data.AlertChannels[i].ID == channel.ID {
preserveSecrets(data.AlertChannels[i].Params, channel.Params)
data.AlertChannels[i] = channel
return s.saveLocked(data)
}
}
data.AlertChannels = append(data.AlertChannels, channel)
return s.saveLocked(data)
}
func preserveSecrets(oldParams, newParams map[string]string) {
for _, key := range []string{"token", "password", "secret", "authorization"} {
if newParams[key] == "" && oldParams[key] != "" {
newParams[key] = oldParams[key]
}
}
}
func (s *FileStore) DeleteAlertChannel(id string) error {
s.mu.Lock()
defer s.mu.Unlock()
data, err := s.loadLocked()
if err != nil {
return err
}
channels := data.AlertChannels[:0]
for _, channel := range data.AlertChannels {
if channel.ID != id {
channels = append(channels, channel)
}
}
data.AlertChannels = channels
for i := range data.Rules {
ids := data.Rules[i].AlertChannels[:0]
for _, channelID := range data.Rules[i].AlertChannels {
if channelID != id {
ids = append(ids, channelID)
}
}
data.Rules[i].AlertChannels = ids
}
return s.saveLocked(data)
}
func (s *FileStore) UpdateRule(id string, fn func(*Rule)) error {
s.mu.Lock()
defer s.mu.Unlock()
data, err := s.loadLocked()
if err != nil {
return err
}
for i := range data.Rules {
if data.Rules[i].ID == id {
fn(&data.Rules[i])
return s.saveLocked(data)
}
}
return errors.New("rule not found")
}
func (s *FileStore) AddIncident(incident Incident) error {
s.mu.Lock()
defer s.mu.Unlock()
data, err := s.loadLocked()
if err != nil {
return err
}
if incident.ID == "" {
incident.ID = slug(incident.RuleName) + "-" + incident.CreatedAt.Format("20060102150405")
}
data.Incidents = append([]Incident{incident}, data.Incidents...)
if len(data.Incidents) > 200 {
data.Incidents = data.Incidents[:200]
}
return s.saveLocked(data)
}
func (s *FileStore) loadLocked() (Data, error) {
var data Data
contents, err := os.ReadFile(s.path)
if errors.Is(err, os.ErrNotExist) {
return data, nil
}
if err != nil {
return data, err
}
if len(contents) == 0 {
return data, nil
}
return data, json.Unmarshal(contents, &data)
}
func (s *FileStore) saveLocked(data Data) error {
if err := os.MkdirAll(filepath.Dir(s.path), 0o700); err != nil {
return err
}
contents, err := json.MarshalIndent(data, "", " ")
if err != nil {
return err
}
return os.WriteFile(s.path, append(contents, '\n'), 0o600)
}
func slug(value string) string {
value = strings.ToLower(strings.TrimSpace(value))
var b strings.Builder
lastDash := false
for _, r := range value {
ok := r >= 'a' && r <= 'z' || r >= '0' && r <= '9'
if ok {
b.WriteRune(r)
lastDash = false
continue
}
if !lastDash {
b.WriteByte('-')
lastDash = true
}
}
return strings.Trim(b.String(), "-")
}

View File

@ -0,0 +1,38 @@
package store
import "testing"
func TestUpsertRuleDefaults(t *testing.T) {
s := NewFileStore(t.TempDir() + "/config.json")
if err := s.UpsertRule(Rule{Name: "API Errors", LogQL: `{service="api"} |= "error"`}); err != nil {
t.Fatal(err)
}
data, err := s.Load()
if err != nil {
t.Fatal(err)
}
if len(data.Rules) != 1 {
t.Fatalf("expected one rule, got %d", len(data.Rules))
}
rule := data.Rules[0]
if rule.ID != "api-errors" || rule.Threshold != 1 || rule.Window != "5m" || rule.Cooldown != "1h" {
t.Fatalf("unexpected defaults: %#v", rule)
}
}
func TestUpsertAlertChannelPreservesSecret(t *testing.T) {
s := NewFileStore(t.TempDir() + "/config.json")
if err := s.UpsertAlertChannel(AlertChannel{ID: "ntfy", Name: "ntfy", Type: "ntfy", Enabled: true, Params: map[string]string{"token": "secret", "topic": "alerts"}}); err != nil {
t.Fatal(err)
}
if err := s.UpsertAlertChannel(AlertChannel{ID: "ntfy", Name: "ntfy", Type: "ntfy", Enabled: true, Params: map[string]string{"topic": "alerts2"}}); err != nil {
t.Fatal(err)
}
data, err := s.Load()
if err != nil {
t.Fatal(err)
}
if got := data.AlertChannels[0].Params["token"]; got != "secret" {
t.Fatalf("secret was not preserved: %q", got)
}
}

View File

@ -0,0 +1,285 @@
# Phase 1 Plan
Goal: make log monitoring useful in production without enabling dangerous automation.
Phase 1 targets a production Portainer deployment of Log Guardian. Loki will likely run on a separate server and use an object-storage backend, probably MinIO backed by NAS storage. Log Guardian must remain safe-by-default: authenticated UI, real alert delivery, incident cooldowns, Loki validation tooling, dry-run remediation evidence, and local-only AI analysis.
## Decisions made
- Deployment target: production Portainer.
- Loki location: separate server from Log Guardian.
- Loki storage direction: object storage via MinIO, with MinIO data on NAS-backed storage.
- UI auth: single admin user using basic auth.
- Health endpoint: `/healthz` remains unauthenticated.
- First alert channel: self-hosted ntfy.
- Additional alert channel types should be supported/configurable from UI, not ntfy-only.
- Incident cooldown: per rule, default `1h`.
- AI: local-only Ollama is approved; no external/cloud AI providers.
- Remediation: Phase 1 should implement permission modeling and dry-run evidence, not dangerous automatic actions.
- Phase 1 completion requires Docker build and real alert delivery validation.
## 1. Production deployment architecture
### Log Guardian
Run Log Guardian as a Portainer-managed service/container.
Required mounts/config:
- Persistent local config/data file mount for `LOG_GUARDIAN_DATA_PATH`.
- Environment configuration for Loki endpoint, auth, alert defaults, and optional Ollama endpoint.
- Network path to the remote Loki server.
- Optional network path to self-hosted ntfy, Gotify, SMTP relay, webhook targets, and Ollama.
### Loki
Loki will run on a different server. Recommended production direction:
- Loki single-binary or simple scalable deployment depending on expected volume.
- Object storage backend: S3-compatible MinIO.
- MinIO data stored on NAS-backed storage.
- Retention configured explicitly.
- Promtail or Grafana Alloy deployed on every Docker/Portainer host that needs logs collected.
Recommended initial storage sizing:
- Start with 50 GB usable object storage for modest private infrastructure.
- Use 14-day retention initially.
- Monitor ingestion and object growth for 7 days before increasing retention.
- Increase to 100-250 GB+ if services are noisy or retention must be longer.
Planning assumptions to validate:
- NAS latency is acceptable for MinIO/Loki workload.
- MinIO has backups/snapshots appropriate for desired retention.
- Loki retention deletes old chunks/index data correctly.
- Log Guardian can reach Loki over a private network/VPN only.
## 2. Loki ingestion, label discovery, and validation
Because actual labels are not known yet, Phase 1 must include label discovery and query testing.
### Collection strategy
Use Promtail or Grafana Alloy on Docker/Portainer hosts. Target labels should include stable, low-cardinality dimensions such as:
- `host`
- `container`
- `service`
- `stack`
- `stream`
- `compose_project` or equivalent
- `compose_service` or equivalent
Avoid high-cardinality labels, including request IDs, user IDs, IP addresses, paths, error messages, or raw log content.
### Log Guardian Loki diagnostics
Add authenticated UI/API support for:
- Loki readiness/status check.
- List labels from `/loki/api/v1/labels`.
- List values for a selected label from `/loki/api/v1/label/{name}/values`.
- Test a LogQL query before saving a rule.
- Show count and sanitized sample lines for a test query.
Implementation notes:
- Limit query time range and result count.
- Do not log raw sample payloads by default.
- Redact obvious secrets from displayed query samples where practical.
- Handle Loki tenant ID and basic auth from existing Loki config.
## 3. UI authentication
Add single-admin basic auth.
Configuration:
- `LOG_GUARDIAN_AUTH_USERNAME`
- `LOG_GUARDIAN_AUTH_PASSWORD`
Rules:
- `/healthz` is unauthenticated.
- All other UI/API routes require auth.
- Use constant-time comparison for credentials.
- Do not log usernames/passwords or auth headers.
- If credentials are unset, fail closed in production or clearly mark auth as disabled only for local development. Prefer fail closed for Portainer deployment.
## 4. Alert channels
Alert channels must be configurable from the UI and stored in local configuration. Secrets need careful handling.
### Required first channel: self-hosted ntfy
Support fields:
- Name
- Enabled
- Server URL
- Topic
- Optional bearer token or token file reference
- Priority
- Tags
Validation:
- Send a test notification from the UI.
- Record delivery result without logging token values.
### Additional Phase 1 channel types
Implement the data model and UI so multiple channel types are possible. Preferred Phase 1 support order:
1. `ntfy` - implement real delivery first.
2. `gotify` - real delivery if time allows; token-based API is straightforward.
3. `generic_webhook` - useful but must avoid leaking secrets; support explicit headers carefully.
4. `smtp` - useful, but can be deferred if it complicates TLS/auth validation.
Minimum acceptable Phase 1 scope:
- UI can create/edit/delete alert channels.
- UI can attach channels to rules.
- ntfy real delivery works.
- Other channel types may be present as disabled/scaffolded if not implemented, but the UI should clearly show unsupported status.
Secret handling:
- Do not print secrets in logs.
- Do not render stored secret values back into forms; show only `configured`/`not configured`.
- Prefer environment variable or mounted file references for sensitive channel tokens when possible.
- If secrets are stored in local JSON, the file must be written with `0600` permissions and documented as sensitive.
## 5. Rule evaluation and incident cooldown
Add per-rule cooldown.
Rule field:
```json
"cooldown": "1h"
```
Default behavior:
- If unset, cooldown is `1h`.
- If a rule fires outside cooldown:
- create incident;
- send configured alert channels;
- record dry-run remediation evidence if actions are configured.
- If a rule fires during cooldown:
- update rule last checked/matched/count fields;
- do not create a duplicate incident;
- do not send another alert;
- increment or record suppressed count if implemented.
## 6. Remediation/action model
Phase 1 should not execute destructive remediation. Implement guardrails and dry-run evidence.
Recommended Phase 1 action types:
- `generic_webhook`: dry-run by default; real execution can be deferred or restricted.
- `portainer_restart`: dry-run only in Phase 1.
- `allowlisted_command`: dry-run only in Phase 1; command IDs only, never arbitrary shell from UI/AI.
- `record_recommendation`: safe action that stores an advisory recommendation on the incident.
Guardrails:
- Global dry-run defaults true.
- Per-rule action must be enabled.
- Per-action dry-run should be explicit or inherited from global dry-run.
- AI may recommend actions but cannot execute them.
- No arbitrary command execution.
- No Portainer write API calls in Phase 1 unless separately approved.
Validation evidence:
- At least one rule with a configured dry-run action.
- Incident shows what would have happened.
- Logs do not expose secrets or raw full log payloads.
## 7. Local-only AI analysis
Local-only Ollama is approved for Phase 1. External/cloud AI providers are not approved.
Configuration:
- `LOG_GUARDIAN_ANALYSIS_PROVIDER=none|ollama`
- `LOG_GUARDIAN_OLLAMA_URL=http://ollama:11434`
- `LOG_GUARDIAN_OLLAMA_MODEL=<model>`
Default:
- `LOG_GUARDIAN_ANALYSIS_PROVIDER=none`
Rules:
- AI analysis remains opt-in per rule.
- AI findings are advisory only.
- Limit number of log lines sent to Ollama.
- Redact obvious secrets before prompt construction.
- Do not log prompts or raw log payloads by default.
- Store only concise summaries/recommendations in incidents.
Prompt/redaction design must be documented before enabling Ollama in production.
## 8. UI scope
Add UI support for:
- Login-protected dashboard.
- Rule create/edit with cooldown and alert channel selection.
- Alert channel create/edit/test.
- Loki status/label discovery/query test.
- Incident list showing alert delivery state and remediation dry-run evidence where available.
Keep the UI served by Go. Do not add a server-side JavaScript runtime.
## 9. Tests and validation
Required implementation tests:
- Auth middleware unit tests.
- Store tests for alert channels, cooldown fields, and secret-preserving updates.
- Loki client tests using `httptest` fixtures for labels, label values, and query range.
- Alert dispatcher tests for ntfy using `httptest`.
- Rule engine tests for cooldown suppression.
- Remediation dry-run tests.
- Optional Ollama provider tests using `httptest` only; no live AI dependency in unit tests.
Required validation before Phase 1 completion:
```sh
gofmt -w cmd internal
go test ./...
make validate-structure
make docker-build
```
Additional required evidence:
- Loki connectivity/query validation against the remote Loki server.
- Label discovery output documented without secrets.
- Self-hosted ntfy test alert received.
- A rule-generated ntfy alert received.
- Remediation dry-run evidence captured.
- Security/privacy review covering logs, prompts, secrets, alert tokens, and action permissions.
## 10. Required Phase 1 artifacts
Create/update these before declaring Phase 1 complete:
- `project-docs/status/phase-1-summary.md`
- `project-docs/status/phase-1-test-results.md`
- `project-docs/status/phase-1-open-issues.md`
## 11. Open planning items
- Final Loki topology: single-binary vs simple scalable.
- Final MinIO/NAS capacity and retention target after observing ingestion.
- Whether Gotify, generic webhook, or SMTP are implemented fully in Phase 1 or scaffolded after ntfy.
- Whether alert channel secrets are stored directly in local JSON or referenced via env/file paths.
- Exact Ollama model to use once Ollama is stable.
- Whether Log Guardian reaches Loki/ntfy/Ollama over LAN, VPN, or Docker overlay network.

View File

@ -0,0 +1,29 @@
# Loki Setup Runbook
## Starter local/Portainer stack
Use `deploy/loki/docker-compose.yml` and `deploy/loki/promtail-config.yml`.
```sh
cd deploy/loki
docker compose up -d
```
## Verify Loki
```sh
curl -fsS http://localhost:3100/ready
curl -G http://localhost:3100/loki/api/v1/labels
```
## Useful LogQL examples
```logql
{service="api"} |= "error"
{stack="second-brain"} |~ "(?i)(panic|fatal|exception)"
{stream="stderr"} |~ "(?i)(error|failed|timeout)"
```
## Portainer notes
For Portainer deployment, mount the Docker socket and Docker container log directory read-only into Promtail. The included compose does this for a single Docker host. Multi-node Swarm setups need Promtail on each node or another log shipping strategy.

View File

@ -0,0 +1,8 @@
# Phase 0 Open Issues
- Add UI authentication before exposing beyond a trusted private network.
- Validate the Loki + Promtail setup against actual Portainer/Docker hosts.
- Decide alert channels: email, ntfy, Gotify, Discord, Matrix, SMS, or another destination.
- Decide which remediation actions are allowed. Candidates: send webhook, restart Portainer service, scale service, pause updater, open issue, or run a tightly allowlisted command.
- AI analysis provider is intentionally unset. Any external AI/cloud provider requires explicit approval and prompt/log redaction design.
- Add real alert delivery, retry behavior, deduplication, silences, and escalation policies.

View File

@ -0,0 +1,16 @@
# Phase 0 Summary
Created initial `log-guardian` scaffold.
Included:
- Go HTTP service with health endpoint and basic web UI.
- Local JSON-backed rule, alert-channel, and incident storage.
- Loki query client for LogQL range queries.
- Rule engine with thresholds and incident creation.
- No-op analyzer interface for future AI agent analysis.
- Dry-run remediation action runner scaffold.
- Alert dispatcher scaffold that logs alert intent only.
- Dockerfile, Portainer stack, Loki + Promtail starter compose, example environment, Makefile, CI workflow, and phase artifacts.
Phase 0 does not certify production alerting or remediation.

View File

@ -0,0 +1,19 @@
# Phase 0 Test Results
Validation run from `/data/code/log-guardian` after scaffold creation.
Passed:
```sh
gofmt -w cmd internal
go test ./...
make validate-structure
make go-fmt
make docker-build
```
Evidence:
- `go test ./...` passed for all packages, including `internal/analysis` and `internal/store` tests.
- `make validate-structure` printed `Structure validation passed`.
- Docker image built successfully as `gitea.wayfinderak.com/wayfinderak/log-guardian:local`.

View File

@ -0,0 +1,10 @@
# Phase 1 Open Issues
- Live deployment validation is still required against production Portainer, remote Loki, and self-hosted ntfy.
- Loki topology is not finalized: single-binary vs scalable deployment, exact MinIO/NAS capacity, and retention settings need confirmation.
- Promtail/Grafana Alloy labels must be discovered on the real Docker/Portainer hosts.
- Gotify, generic webhook, and SMTP channels are scaffolded but not implemented for real delivery.
- Alert channel secrets are currently stored in the local JSON config if entered through the UI; mounted secret-file/env references should be considered for stronger secret handling.
- Ollama provider is implemented as local-only and opt-in, but the final model and runtime deployment still need validation.
- No destructive remediation is implemented; Portainer restart and allowlisted command actions are dry-run evidence only.
- The UI supports create/replace workflows but does not yet provide polished edit forms for existing rules/channels.

View File

@ -0,0 +1,16 @@
# Phase 1 Summary
Implemented Phase 1 foundations for production use:
- Basic auth protects UI/API routes; `/healthz` remains unauthenticated.
- Loki diagnostics endpoints were added for labels, label values, and limited query testing.
- Rules now support per-rule cooldowns with default `1h` and duplicate incident suppression.
- Alert channels are configurable in the UI.
- Real self-hosted ntfy delivery is implemented with test-send support.
- Gotify, generic webhook, and SMTP channel types are represented as scaffolded options.
- Incidents can record alert delivery results and remediation dry-run evidence.
- Remediation action runner supports guarded dry-run evidence for generic webhook, Portainer restart, allowlisted command, and recommendation actions.
- Local-only Ollama analysis support was added and remains disabled by default.
- Secret handling was improved for alert channel token preservation and display masking.
Phase 1 production rollout still requires live validation against the user's remote Loki and self-hosted ntfy deployments.

View File

@ -0,0 +1,37 @@
# Phase 1 Test Results
Validation run from `/data/code/log-guardian` after Phase 1 implementation.
Passed:
```sh
gofmt -w cmd internal
go test ./...
make validate-structure
make go-fmt
make go-test
make docker-build
```
Docker image built successfully as:
```text
gitea.wayfinderak.com/wayfinderak/log-guardian:local
```
Unit/integration-style tests added:
- Basic auth middleware protects UI routes and leaves `/healthz` unauthenticated.
- Store defaults include rule cooldown and alert channel secret-preserving updates.
- Loki client label, label-value, and query-range behavior uses `httptest` fixtures.
- ntfy dispatcher delivery uses an `httptest` fixture.
- Rule engine suppresses duplicate incidents during cooldown.
Pending live production validation before Phase 1 can be called operationally complete:
- Remote Loki connectivity and label discovery.
- Remote Loki LogQL query validation.
- Self-hosted ntfy test notification received.
- Rule-generated ntfy alert received.
- Remediation dry-run evidence captured from a real rule.
- Security/privacy review of logs, prompts, secrets, alert tokens, and action permissions.

1
web/static/app.css Normal file
View File

@ -0,0 +1 @@
:root{color-scheme:dark;--bg:#0d1117;--panel:#161b22;--text:#f0f6fc;--muted:#8b949e;--accent:#58a6ff;--danger:#ff7b72}*{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--text);font-family:system-ui,-apple-system,Segoe UI,sans-serif}.shell{max-width:1200px;margin:0 auto;padding:2rem}.card{background:var(--panel);border:1px solid #30363d;border-radius:16px;padding:1rem;margin:1rem 0}.grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(220px,1fr));gap:1rem;align-items:end}.wide{grid-column:1/-1}label{display:grid;gap:.35rem;color:var(--muted)}.check{display:flex;gap:.5rem;align-items:center}input{width:100%;padding:.65rem;border-radius:10px;border:1px solid #30363d;background:#010409;color:var(--text)}button{padding:.65rem .9rem;border:0;border-radius:10px;background:var(--accent);color:#06111f;font-weight:700;cursor:pointer}.row{display:flex;justify-content:space-between;gap:1rem;align-items:center}table{width:100%;border-collapse:collapse}th,td{text-align:left;padding:.7rem;border-top:1px solid #30363d;vertical-align:top}code{color:#a5d6ff}.muted{color:var(--muted)}.error{color:var(--danger)}@media(max-width:760px){.shell{padding:1rem}table{display:block;overflow-x:auto}}