Operating

Privacy & security

Privacy is the product. These are hard rules, not best-effort goals.

Hard rules

No source upload by default. MCP sends metadata only.
No PAT storage. Auth uses GitHub Device Flow.
No wallet or hotkey display.
No raw trust-score display.
No payout/reward guarantees, anywhere.
No farming language.
No public score estimates.
No private reviewability details in public GitHub output.

Open algorithm, private tuning

LoopOver's review engine is built so the logic is public but the dial settings are not. The deterministic gate, the scoring signals, the slop detector, the grounding/RAG context builders, and the comment renderer all live in the open source tree — anyone can read exactly how a verdict is reached. What stays private is the production tuning: the thresholds, guardrail paths, and gate modes an operator runs in production. That separation is what keeps a review from being gameable off the public code.

Tuning lives in two private, repo-scoped places that sit on top of the open algorithm, and neither reveals review direction:

Per-repo settings — gate modes, score thresholds, and guardrails, stored in the operator's database (set through the dashboard/API) or declared as config-as-code in a repo's .loopover.yml. Choosing gate.slop.minScore or setting settings.hardGuardrailGlobs tightens the gate without telling a contributor how to bypass it.
Operator feature flags — the LOOPOVER_REVIEW_* family of worker environment variables. These switch whole capabilities (safety scanning, CI and full-file grounding, RAG context, reputation-based spend control, the unified comment) on or off for a deployment.

Every feature flag ships OFF, and a per-PR capability runs only when its own flag is on and the repo is in the LOOPOVER_REVIEW_REPOS allowlist — so capabilities stay dormant until an operator explicitly converges a repo, one flag and one repo at a time.

# Per-PR features run only when the flag is ON and the repo is allowlisted.
LOOPOVER_REVIEW_REPOS="JSONbored/loopover"   # per-repo cutover allowlist (default: none)
LOOPOVER_REVIEW_SAFETY="true"                  # prompt-injection defang + secret-leak scan
LOOPOVER_REVIEW_GROUNDING="true"               # CI status + full changed-file content
LOOPOVER_REVIEW_RAG="true"                     # codebase vector-index context (needs index)
LOOPOVER_REVIEW_IMPACT_MAP="true"              # deterministic impact map (needs review.impact_map too)
LOOPOVER_REVIEW_CULTURE_PROFILE="true"         # repo quality-culture profile (needs review.culture_profile: true)
LOOPOVER_REVIEW_MEMORY="true"                  # repeat-false-positive suppression (needs review.memory too)
LOOPOVER_REVIEW_REPUTATION="true"              # submitter-reputation spend control (never shown)
LOOPOVER_REVIEW_ENRICHMENT="true"              # external analyzer registry (REES) findings
LOOPOVER_REVIEW_INLINE_COMMENTS="true"         # diff-anchored inline PR review comments
LOOPOVER_REVIEW_FIX_HANDOFF="true"             # machine-readable fix-handoff block (contributor-run)
LOOPOVER_REVIEW_PLANNER="true"                 # @loopover plan on-demand implementation plan
LOOPOVER_REVIEW_SCREENSHOTS="true"             # before/after visual capture for UI changes
LOOPOVER_REVIEW_E2E_TESTS="true"               # AI-generated E2E test coverage (needs features.e2eTests too)
LOOPOVER_REVIEW_IMPROVEMENT_SIGNAL="true"      # read-only PR quality-delta signal (activation only, no-op for now)
LOOPOVER_REVIEW_AMS_REPUTATION_BRIDGE="true"   # upgrade-only AMS track-record bridge (never shown; needs features.amsReputationBridge + LOOPOVER_AMS_TRACK_RECORD_URL)

# Global (cron / endpoint) flags, not scoped by LOOPOVER_REVIEW_REPOS.
LOOPOVER_REVIEW_CONTINUOUS="true"              # fleet-wide default: re-review on every push (else one-shot)
LOOPOVER_REVIEW_OPS="true"                     # read-only anomaly scan + outcome stats endpoint
LOOPOVER_REVIEW_SELFTUNE="true"                # self-tightening tuning loop, never loosens
LOOPOVER_REVIEW_PARITY_AUDIT="true"            # shadow-record gate-decision parity readiness
LOOPOVER_REVIEW_CONTENT_LANE="true"            # dedicated content/registry-repo review lane
LOOPOVER_REVIEW_DRAFT="true"                   # public draft-submission (contributor fork PR) flow

The internal-only controls never surface publicly. Submitter reputation, for example, can downgrade a burst or low-reputation submitter to a deterministic-only review — but no comment, label, or check ever shows a reputation value. Reputation thresholds are generic anti-abuse defaults that reveal no review direction and are not per-repo tunable.

Reading the open source tells you how a verdict is computed, never what an operator's production gate will decide. The deciding inputs — thresholds, guardrail globs, and which LOOPOVER_REVIEW_* capabilities are live — are private runtime settings, so reviews cannot be reverse-engineered or gamed from the public code.

Public output rules

At most one sticky sanitized comment per confirmed-miner PR.
At most one configured label per confirmed-miner PR.
Public comments are maintainer-friendly and non-shaming.

Auth

Public endpoint: GET /health.
Private API uses Bearer / session tokens.
MCP CLI uses GitHub OAuth Device Flow.
Static bearer tokens remain internal / bootstrap only.

Website copy may discuss private scoreability and risk reasoning, but it's always framed as private MCP/API context. The public web never carries score numbers.

Orb fleet telemetry (#4893)

The always-on fleet-calibration pipeline (POST /v1/orb/ingest) aggregates anonymized signals from every registered and unregistered self-host instance into one shared table (orb_signals) — see self-hosting: GitHub App for what's exported today and how to opt out. That table is intentional and not a leak: repo/PR identifiers are HMAC-anonymized with a per-instance secret LoopOver's own collector never holds, and nothing beyond verdict/outcome/reversal/cycle-time is even read out of an instance's local data before export.

This section is the forward-looking half the code doesn't yet need to answer: as the product grows a hosted, per-customer model, what would be safe to add to this pipeline, and what wouldn't? A clear yes/no per category, so a future PR that touches this table has a rule to check itself against instead of relitigating anonymization from scratch.

Potential future data	Safe to add?	Why
Raw repo/org names or logins, even opt-in	No	The anonymization promise is fleet-wide, not per-tenant-configurable — one customer opting into raw identifiers doesn't change what every other instance was told about this shared table. A hosted customer who wants their own data identifiable should read it back from their own instance/dashboard, never from the shared fleet table.
PR/commit content, diffs, or comments	No	Never queried into the export today (the collector's own DB query selects only verdict/outcome/reversal/timestamps) — there is no path for this to leave an instance short of a deliberate, separate feature. Adding it to fleet telemetry would be a new capability, not an extension of this one.
Full-text gate reasons	No	Already bucketed to a fixed, low-cardinality category before export specifically so free text (which can quote file paths, identifiers, or reviewer commentary) never leaves the instance. A hosted model doesn't change the fact that this text wasn't written for cross-tenant eyes.
A raw `installation_id` column on `orb_signals` itself	No	`orb_signals`/`orb_instances` were built with only an opaque, HMAC-derived `instance_id` — retrofitting a raw installation id onto that table would retroactively de-anonymize every row collected under the existing anonymity promise. If hosted billing/support ever needs to map a fleet instance to a paying installation, that mapping belongs in a separate table (mirroring how `orb_github_installations` already keeps raw account identity in its own table, entirely apart from `orb_signals`), not a new column here.
Per-instance metrics surfaced back to that instance's own operator	Yes, with new authz	`GET /v1/internal/fleet/analytics` already computes per-instance breakdowns keyed by the same opaque `instanceId` — the data already exists. A hosted per-customer dashboard could show a customer their own row. What doesn't exist yet is anything that maps "this session belongs to instance X" and enforces that a customer can only ever see their own opaque id, never another tenant's — that authz layer is new work, not a data-model change.
Cross-tenant fleet-median/percentile comparisons (e.g. gate precision vs. the peer median)	Yes, unchanged	Already shipped, in aggregate form only — a single percentile/median across the whole fleet, with no way to reverse-engineer any other specific tenant's raw value from it. This category is the one place "more data" is already safe, because it was designed to stay aggregate-only from the start.

A real gap, not a future one

Anonymization is enforced entirely on the sender, not the receiver: ORB_ANONYMIZE=false makes a self-host instance export raw repo/PR names, and POST /v1/orb/ingest has no server-side check that a submitted repo_hash/pr_hash actually looks like a hash before storing it in the shared table — it validates only type and length. Today this only affects an operator who deliberately opted out for their own private collector. It becomes a sharper problem the moment /v1/orb/ingest is fronted by anything resembling a hosted, multi-tenant path: a misconfigured or malicious sender could land raw identifiers in a table every other instance was told stays anonymized. Closing this (reject a submission whose repo_hash/pr_hash doesn't match the expected HMAC-output shape) is real follow-up work, tracked separately from this review.

← PreviousTuning your reviews Next →Troubleshooting