OMEGA Health: At risk

Generated 2026-06-22T21:24:40Z. Scenario automated-test-run:0h:0m.

Latest omega-automated-test-run-2026-06-22t19-37-16z Baseline none Projects 1 Data beta-2026-06-22t21-24... AI not run
Current Project1This page is scoped to this project only.
omega OMEGA mixed | 424 files | 8 inputs | 3 open todos
BETA Action Output 
Ready.
Data Authenticity Real data gate excluded demo inputs

Metrics use only accepted real reports; quarantined reports remain visible for audit. Only accepted real reports are used for metrics, graphs, claims, progress, and AI analysis. Synthetic/demo/local proof reports are quarantined.

Real Used2drives charts
Quarantined12excluded
Discovered14total reports
  • run_id contains 'local-proof'6 report(s)
  • scenario_id contains 'coastal-demo'12 report(s)
  • scenario_key contains 'coastal-demo'12 report(s)
  • source_path contains 'local-proof'6 report(s)
  • metadata.database_path contains 'local-proof'6 report(s)
quarantined
baseline

Create a comparable baseline before judging progress.

The dashboard needs a matching prior scenario before improvement claims are meaningful.

Data quality 38% thin

Focus Metrics

  • No metric regressionsKeep collecting repeated runs and broader scenarios.

Next Moves

  • Repeatable Ingest Load Curve P1 | Backend/performance
  • Endpoint Latency Distribution P1 | Gateway/API
  • Storage Growth Curve P1 | Data/storage
  • Firmware Boot And I/O Trace P1 | Firmware

Missing Inputs

  • Repeatability Run the same scenario several times before treating a change as proven.
  • Sample Scale Run proof scenarios at 100, 500, and 1000+ observations and compare the curves.
  • Source Connection Plan Register planned sources, then connect real files or folders as they become available.
  • Comparable Baseline Repeat the same scenario after changes so every metric has a before/after comparison.

Real Evidence Capture Kit

OMEGA has real evidence; repeat matching scenarios to prove improvement.

collect-repeat-baseline

Starter Scenarios

benchP1
Bench Validation Baseline

Creates the first accepted real baseline so charts and claims stop being empty.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId bench-validation -CollectionType bench .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId bench-validation -CollectionType bench -RequiredPassed -AdvisoryPassed -ObservationsAccepted 100 One measured report imports as real, required gates pass, and the dashboard shows one real run.
fieldP1
Field-Link Load Check

Connects portal/API responsiveness to real network conditions instead of local-only timing.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId field-link-load -CollectionType field .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId field-link-load -CollectionType field -RequiredPassed -ObservationsAccepted 100 -PortalHtmlBytes 1 -QueryP95Ms 1 Portal payload and query p95 are recorded from a constrained or remote path.
ciP1
CI Regression Evidence

Adds build/test pass history so project progress is not inferred only from proof runs.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId ci-regression -CollectionType ci .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId ci-regression -CollectionType ci -RequiredPassed -ObservationsAccepted 1 CI result evidence is attached and repeat failures trend down over time.
firmwareP1
Firmware Boot And I/O Trace

Firmware projects need boot and sensor/I/O evidence before readiness claims are trusted.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId firmware-boot-io -CollectionType firmware .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId firmware-boot-io -CollectionType firmware -RequiredPassed -ObservationsAccepted 1 Serial boot logs or firmware checks are summarized in the report evidence notes.
firmwareP1
Firmware Identity, Health, And OLED Evidence

Connects the new OMEGA firmware identity, board-lineage, OLED, and health telemetry fields to real evidence instead of code-only claims.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId firmware-identity-health -CollectionType firmware .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId firmware-identity-health -CollectionType firmware -RequiredPassed -ObservationsAccepted 1 -Note "Capture serial heartbeat and OLED evidence for operator_label physical_board_tag physical_mac esp_temp_c battery fields" Serial/JSON/OLED evidence shows operator_label, physical_board_tag, physical_mac, mesh/node identity, esp_temp_c, battery/PMU health where supported, and readable OLED state.

Required Report Fields

  • metadata.data_authenticityrealLets BETA accept the report as usable evidence instead of quarantining it.
  • metadata.collection_typebench | field | ci | firmware | hardware | operator | measuredTells BETA what kind of real-world source produced the measurement.
  • scenario.scenario_idstable scenario idMatching scenario ids allow before/after comparisons over time.
  • overall.required_passedtrue/falseCorrectness gates are the minimum proof before performance claims matter.
  • metrics.ingest.counts.observations.acceptedintegerSample size affects confidence in throughput, storage, and latency claims.
  • analysis.evidence[].summaryshort measured source noteKeeps the metric tied to the actual test, bench note, CI run, or field observation.
  • analysis.evidence[].sourcelog/photo/serial/CI/bench source referenceLets a reviewer trace the metric back to the file, capture, or note that produced it.

Metric Field Map

MeasurementReport FieldPriority
Repeatable Ingest Load Curveobservation_throughput_per_s metrics.ingest.observation_throughput_per_s P1
Endpoint Latency Distributionquery_p95_ms metrics.queries.all_queries_ms.p95_ms P1
Storage Growth Curvedb_bytes_per_observation metrics.efficiency.db_bytes_per_observation P1
Field-Link Portal Loadportal_html_bytes metrics.coverage.portal_html_bytes P2
Reliability And Failure Historyfailure_rate analysis.evidence[] or imported CI input P2
Human Acceptance Signoffoperator_acceptance analysis.evidence[] or field/operator signoff P2
Firmware Boot And I/O Tracefirmware_boot_success_rate analysis.evidence[] or firmware report P1
Firmware Identity And Board Lineagefirmware_identity_fields_present analysis.evidence[] or firmware JSON/serial frame P1
Firmware Power And Thermal Healthfirmware_health_telemetry_present analysis.evidence[] or firmware JSON/serial frame P1
Operator OLED Readabilityoperator_oled_readability analysis.evidence[] or operator/OLED inspection note P2
Required Gate Pass Raterequired_gate_pass_rate metrics.custom.required_gate_pass_rate P1
Advisory Gate Pass Rateadvisory_gate_pass_rate metrics.custom.advisory_gate_pass_rate P1
Wire Savingswire_savings_percent metrics.wire.savings_percent P1
Unit Test Pass Rateunit_test_pass_rate metrics.custom.unit_test_pass_rate P1
Test Failurestest_failure_count metrics.custom.test_failure_count P1
Test Pass Ratetest_pass_rate analysis.evidence[] or imported CI input P2
Overall Health62%

At risk. Weighted from proof, coverage, backlog, and regressions.

Needs baseline
Proof Score61%

Correctness, effectiveness, and efficiency score.

Evidence Coverage17%

2 of 12 evidence signals present.

Data Quality38%

thin. Baselines, repeatability, sample size, and AI review.

Backlog Health100%

Falls as high-priority findings and open work increase.

What This Is Tracking

BETA compares the latest accepted real proof report against the previous real run with the same scenario. Quarantined demo data is listed for audit but never drives health, progress, claims, or AI analysis.

Correctnesspassing
35.00 / 40.0

Required and advisory proof gates.

Effectivenessthin
7 / 35.0

Observations, stations, bathymetry, mesh routes, and acoustic messages.

Efficiencythin
19.00 / 25.0

Read/write p95 latency, portal payload, storage cost, and wire savings.

Progress Trends Score Throughput Query p95
Latest Run Comparison Observation ingest throughput n/a Write p95 n/a Query p95 n/a Station features n/a Bathymetry points n/a Mesh routes n/a Portal payload n/a DB bytes per observation n/a Wire savings n/a Required Gate Pass Rate n/a Advisory Gate Pass Rate n/a Unit Test Pass Rate n/a Test Failures n/a

Progress Over Time

Create a matching baseline before judging progress.

Run Timeline Score Improved Regressed

Metric Drilldowns

MetricLatestPreviousStatusBestWhy
Observation ingest throughputMeasures gateway ingest speed on the proof scenario 0 obs/s none no-baseline 0 obs/somega-automated-test-run-2026-06-22t19-37-16z Observation ingest throughput needs a matching baseline before progress can be judged.
Write p95Captures ingest responsiveness without being fooled by only average latency. 0 ms none no-baseline 0 msomega-automated-test-run-2026-06-22t19-37-16z Write p95 needs a matching baseline before progress can be judged.
Query p95Measures portal/API query responsiveness under the proof scenario 0 ms none no-baseline 0 msomega-automated-test-run-2026-06-22t19-37-16z Query p95 needs a matching baseline before progress can be judged.
Station featuresShows that station coverage is present rather than only raw observations. 0 none no-baseline 0omega-automated-test-run-2026-06-22t19-37-16z Station features needs a matching baseline before progress can be judged.
Bathymetry pointsShows that terrain/depth data is included in the proof evidence. 0 none no-baseline 0omega-automated-test-run-2026-06-22t19-37-16z Bathymetry points needs a matching baseline before progress can be judged.
Mesh routesShows whether route/network behavior is being exercised. 0 none no-baseline 0omega-automated-test-run-2026-06-22t19-37-16z Mesh routes needs a matching baseline before progress can be judged.
Portal payloadLarge payloads hurt slow-link and tunnel usability. 0 B none no-baseline 0 Bomega-automated-test-run-2026-06-22t19-37-16z Portal payload needs a matching baseline before progress can be judged.
DB bytes per observationShows storage efficiency and whether growth is becoming expensive. 0 B/obs none no-baseline 0 B/obsomega-automated-test-run-2026-06-22t19-37-16z DB bytes per observation needs a matching baseline before progress can be judged.
Wire savingsMeasures binary mesh protocol efficiency versus JSON 0 % none no-baseline 0 %omega-automated-test-run-2026-06-22t19-37-16z Wire savings needs a matching baseline before progress can be judged.
Required Gate Pass RatePercent of accepted OMEGA proof reports where required checks pass 0 % none no-baseline 0 %omega-automated-test-run-2026-06-22t19-37-16z Required Gate Pass Rate needs a matching baseline before progress can be judged.
Advisory Gate Pass RatePercent of accepted OMEGA proof reports where advisory performance checks pass 0 % none no-baseline 0 %omega-automated-test-run-2026-06-22t19-37-16z Advisory Gate Pass Rate needs a matching baseline before progress can be judged.
Unit Test Pass RatePercent of OMEGA pytest tests passing 100.0 % none no-baseline 100.0 %omega-automated-test-run-2026-06-22t19-37-16z Unit Test Pass Rate needs a matching baseline before progress can be judged.
Test FailuresFailing OMEGA tests that need fixing 0 count none no-baseline 0 countomega-automated-test-run-2026-06-22t19-37-16z Test Failures needs a matching baseline before progress can be judged.

Run Timeline

RunGeneratedPostureScoreImprovedRegressedFindings
omega-automated-test-run-2026-06-22t19-37-16z 2026-06-22T19:37:16Z baseline 61.00 0 0 0
omega-real-underwater-ops-20260606 2026-06-06T19:05:22Z baseline 93.00 0 0 1

Metrics Used

These are the current gauges. Each one has a direction, a latest value, and, when possible, a baseline value from a comparable run.

MetricLatestBaselineStatusGauge
Observation ingest throughputMeasures gateway ingest speed on the proof scenario 0 obs/s none no-baseline The metric has a real baseline, a repeat run, and a clear decision rule.
Write p95The slow end of write latency; 95 percent of writes were faster than this. 0 ms none no-baseline Lower is better; under 100 ms currently earns efficiency credit.
Query p95Measures portal/API query responsiveness under the proof scenario 0 ms none no-baseline The metric has a real baseline, a repeat run, and a clear decision rule.
Station featuresCount of station features represented in the proof output. 0 none no-baseline Higher is better; nonzero coverage earns effectiveness credit.
Bathymetry pointsCount of bathymetry points available to the scenario. 0 none no-baseline Higher is better; should grow with richer scenarios.
Mesh routesCount of mesh routes represented in the scenario. 0 none no-baseline Higher is better; nonzero routes earn effectiveness credit.
Portal payloadSize of the served portal shell. 0 B none no-baseline Lower is better; below 350 KB currently earns efficiency credit.
DB bytes per observationDatabase size divided by accepted observations. 0 B/obs none no-baseline Lower is better; below 150 KB per observation currently earns efficiency credit.
Wire savingsMeasures binary mesh protocol efficiency versus JSON 0 % none no-baseline The metric has a real baseline, a repeat run, and a clear decision rule.
Required Gate Pass RatePercent of accepted OMEGA proof reports where required checks pass 0 % none no-baseline The metric has a real baseline, a repeat run, and a clear decision rule.
Advisory Gate Pass RatePercent of accepted OMEGA proof reports where advisory performance checks pass 0 % none no-baseline The metric has a real baseline, a repeat run, and a clear decision rule.
Unit Test Pass RatePercent of OMEGA pytest tests passing 100.0 % none no-baseline Real baseline, repeat run, clear decision rule.
Test FailuresFailing OMEGA tests that need fixing 0 count none no-baseline Failures trend to zero.

Metric Strategy

BETA uses metrics for three jobs: prove the project works, prove it is improving, and decide what work deserves attention next.

compare-and-improve
metric family4
Proof Quality

Separates real evidence from templates, demos, and unsupported claims.

Planning use: Do this first when real_count is zero or reports are quarantined. Examples: real_report_available, required_passed, repeatability, scenario_diversity
metric family4
Effectiveness

Shows whether the build does the thing it claims to do.

Planning use: Use this when deciding whether the core workflow is ready for broader testing. Examples: observations_accepted, station_features, mesh_routes, firmware_identity_fields_present
metric family4
Efficiency

Shows whether the build does the work fast enough and cheaply enough.

Planning use: Use this after correctness is credible, or when field constraints are tight. Examples: observation_throughput_per_s, query_p95_ms, db_bytes_per_observation, portal_html_bytes
metric family4
Reliability

Shows whether good results repeat instead of appearing once.

Planning use: Use this before calling an improvement durable or release-ready. Examples: failure_rate, test_pass_rate, firmware_boot_success_rate, repeat_depth
metric family4
Project Management

Shows whether the work loop is producing useful evidence or wasting time.

Planning use: Use this to decide what to focus on, stop doing, connect, or delegate. Examples: work evidence percent, blocked/rework time, connected sources, planned measurements

Metric Purpose Map

Use this table to understand what each metric proves and what project decision it should drive.

MetricQuestionWhy ImportantPlanning DecisionDone When
Repeatable Ingest Load Curveobservation_throughput_per_s Can the system ingest useful amounts of field data? Proves the system can accept field observations at useful rates without hidden bottlenecks. Use this to decide whether to optimize ingest, batch writes, or reduce input overhead. Throughput holds or improves across repeated matching runs with no failed required gates.
Endpoint Latency Distributionquery_p95_ms Will operators and dashboards get answers quickly? Operator readiness depends on fast reads, not only successful ingest. Use this to prioritize API, database, cache, or payload work. Query p95 is stable or improving, with endpoint-level evidence explaining outliers.
Storage Growth Curvedb_bytes_per_observation Is storage growth controlled as data volume rises? Storage cost must stay bounded as data volume grows beyond tiny demos. Use this to plan schema, retention, compaction, and data-shape work. DB bytes per observation stays flat or drops as sample size grows.
Field-Link Portal Loadportal_html_bytes Can the portal load on slow or remote links? Slow links and tunnels need payload and load-time evidence, not only local browser checks. Use this to prioritize asset size, compression, and route-level payload work. Portal payload and load timing remain within the field-readiness threshold.
Reliability And Failure Historyfailure_rate How often does the project fail or repeat the same problem? Repeated failures and flaky tests are project-management signals, not just engineering annoyances. Use this to decide whether to stabilize before adding scope. Failure rate trends down and high-priority defects do not repeatedly reappear.
Human Acceptance Signoffoperator_acceptance Does a real reviewer or operator trust the result? The proof should connect to whether a real user or reviewer can trust the system. Use this to schedule review, field-test signoff, and handoff work. Each major claim has at least one human-reviewed acceptance record.
Firmware Boot And I/O Tracefirmware_boot_success_rate Does firmware boot and report expected I/O repeatedly? Firmware evidence is needed for devices that must start reliably and read sensors safely. Use this to choose between firmware stabilization, hardware checks, or field tests. Firmware boots repeatedly and reports expected I/O without unsafe failures.
Firmware Identity And Board Lineagefirmware_identity_fields_present Can each frame be traced to the correct physical board? Recent OMEGA firmware emits operator labels, physical board tags, and factory MACs; proof should catch wrong-firmware or wrong-board mistakes. Use this before trusting field telemetry from multiple devices. Every tested firmware target emits stable identity fields and the operator can map each frame to the expected physical board.
Firmware Power And Thermal Healthfirmware_health_telemetry_present Can the node report power and thermal health during operation? Recent firmware adds ESP temperature and PMU/battery fields that prove nodes can be monitored during operation. Use this to decide whether a node is ready for soak, bench, or field work. Health telemetry appears in repeated frames without sentinel or missing values on boards that support those sensors.
Operator OLED Readabilityoperator_oled_readability Can a human identify the board and health state from the device itself? The dense one-screen OLED layout needs human-readable operator evidence, not only code review. Use this to decide whether UI/firmware display work blocks field use. An operator can identify the board and key health state from the OLED in one screen.
Required Gate Pass Raterequired_gate_pass_rate What does Required Gate Pass Rate prove for this project? Percent of accepted OMEGA proof reports where required checks pass Use it to decide whether the next step is testing, fixing, scaling, or stopping. The metric has a real baseline, a repeat run, and a clear decision rule.
Advisory Gate Pass Rateadvisory_gate_pass_rate What does Advisory Gate Pass Rate prove for this project? Percent of accepted OMEGA proof reports where advisory performance checks pass Use it to decide whether the next step is testing, fixing, scaling, or stopping. The metric has a real baseline, a repeat run, and a clear decision rule.
Wire Savingswire_savings_percent What does Wire Savings prove for this project? Measures binary mesh protocol efficiency versus JSON Use it to decide whether the next step is testing, fixing, scaling, or stopping. The metric has a real baseline, a repeat run, and a clear decision rule.
Unit Test Pass Rateunit_test_pass_rate What does Unit Test Pass Rate prove for this project? Percent of OMEGA pytest tests passing Use it to decide whether the next step is testing, fixing, scaling, or stopping. Real baseline, repeat run, clear decision rule.

How To Use Metrics For Planning

  • SignalPick one project goal or claim.
  • SignalChoose the metric that would prove movement toward that goal.
  • SignalCollect one real baseline report or source input.
  • SignalMake one focused change or run one focused bench/field/test cycle.
  • SignalRepeat the same scenario and compare the metric to the baseline.
  • SignalUse the Manager screen to turn the result into the next priority, risk, or stop-doing item.

Project Goals Driving Metrics

  • SignalProve OMEGA gateway proof runs pass required and advisory gates on real measured pipeline data
  • SignalTrack software, firmware, mesh, portal, and field-readiness progress from accepted evidence only
  • SignalUse BETA to identify the next highest-value OMEGA test, source input, and blocker

Data Version & Model Provenance

This records exactly what generated this page, what schemas were used, and whether an Ollama model contributed advisory analysis.

beta-2026-06-22t21-...
Data version beta-2026-06-22t21-24-40z-228d8bfa07 228d8bfa0747582e20b5a087c9598f33458e30e45e51ca2e9ca24ea70864074b
BETA app 0.2.0 Build Environment for Testing & Analytics
Analysis schema beta.analysis.v1 BETA deterministic engine
Generated 2026-06-22T21:24:40Z C:\Users\jdcap\Documents\Projects\BETA\.beta
AI model none AI not used; ollama model none; available=False; usable=False; source=none
Data policy deterministic source of truth Only accepted real reports are used for metrics, graphs, claims, progress, and AI analysis. Synthetic/demo/local proof reports are quarantined.
Real reports 2 Accepted reports used for metrics, graphs, claims, and progress.
Quarantined reports 12 Visible for audit but excluded from proof calculations.

Project Version Records

  • OMEGA Config: v3 (explicit) | Schema: beta.project.v1 Project AI plan: llama3.2:latest | Profile: 2026-06-06T19:08:02Z | Plan: 2026-06-06T19:08:02Z

Data Used

This is the exact source trail behind the evidence screen. Scores are computed only from accepted real proof reports, then enriched with project plans when available.

Real reports used 2 Only accepted real reports are used for metrics, graphs, claims, progress, and AI analysis. Synthetic/demo/local proof reports are quarantined.
Quarantined reports 12 Excluded before metrics, graphs, claims, and AI analysis.
Latest report omega-automated-test-run-2026-06-22t19-37-16z C:\Users\jdcap\Documents\Projects\BETA\.beta\projects\omega\evidence\omega-automated-test-run-2026-06-22t19-37-16z\report.json
Latest generated 2026-06-22T19:37:16Z
Baseline report none No matching scenario baseline yet.
Comparison key automated-test-run:0h:0m The latest proof report is compared with the previous proof report that has the same scenario id, duration, and step size.

Quarantined Inputs

  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\local-proof-smoke\report.json
  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\local-proof-charts\report.json
  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\local-proof-ai\report.json
  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\local-proof-ai2\report.json
  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\local-proof-ai-llama\report.json
  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\local-proof-ai-final\report.json
  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\proof-coastal-demo-20260605130655\report.json
  • Quarantined reportC:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\proof-coastal-demo-20260605130735\report.json

Project Planning Inputs

  • OMEGA Profile: C:\Users\jdcap\Documents\Projects\BETA\.beta\projects\omega\profile.json Plan: C:\Users\jdcap\Documents\Projects\BETA\.beta\projects\omega\plan.json AI plan: C:\Users\jdcap\Documents\Projects\BETA\.beta\projects\omega\ai-plan.json

Claims And Evidence

This is the relevance check: every tracked metric should support a claim that matters to the system being built.

criticalthin
60%

The system accepts and stores valid observations correctly.

This is the foundation: performance and charts do not matter if ingest correctness fails.
criticalgap
20%

Operators can retrieve useful current state quickly.

OMEGA needs to be usable as an operational system, not only as a data sink.
highgap
20%

The proof includes meaningful environmental and route context.

Bathymetry, stations, and routes make the proof relevant to the actual domain.
highgap
20%

The communications path is efficient enough for constrained links.

Wire savings and payload size matter for field/tunnel/low-bandwidth conditions.
highgap
40%

Storage and performance costs are bounded as data grows.

A useful system must stay affordable and responsive beyond tiny demos.

Claim Evidence Reasoning

Each claim below shows the deterministic reasoning chain: source report, signal evidence, metric deltas, and caveats.

The system accepts and stores valid observations correctly. The claim has partial support and should not be treated as fully proven yet. Present signals: Required proof gates and Observation ingest. Missing signals: Advisory proof gates. There is no comparable baseline yet, so metric values describe current state but cannot prove improvement.
thin 60%
Latest omega-automated-test-run-2026-06-22t19-37-16z | Baseline none | Extra report notes 1
  • present Required proof gates Required proof gates is present at 1. Required pass/fail gates are the minimum correctness proof. latest proof report overall gates: 1
  • missing Advisory proof gates Advisory proof gates is missing or zero in the latest report. Add this evidence before relying on the claim. latest proof report overall gates: 0
  • present Observation ingest Observation ingest is present at 2. Accepted observations prove the ingest path handled the scenario data. latest proof report ingest metrics: 2
  • no-baseline Write p95 Write p95 is currently 0 ms, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 ms
  • no-baseline Observation ingest throughput Observation ingest throughput is currently 0 obs/s, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 obs/s
Caveats and gaps
  • Missing signal evidence: Advisory proof gates.
  • No comparable baseline exists for this claim yet.
  • Evidence score is below the adequate threshold.
Operators can retrieve useful current state quickly. The claim has a clear evidence gap. Present signals: none. Missing signals: Read latency, Station coverage, and Portal payload. There is no comparable baseline yet, so metric values describe current state but cannot prove improvement.
gap 20%
Latest omega-automated-test-run-2026-06-22t19-37-16z | Baseline none | Extra report notes 1
  • missing Read latency Read latency is missing or zero in the latest report. Add this evidence before relying on the claim. latest and baseline proof report query metrics: 0 ms
  • missing Station coverage Station coverage is missing or zero in the latest report. Add this evidence before relying on the claim. latest proof report coverage metrics: 0
  • missing Portal payload Portal payload is missing or zero in the latest report. Add this evidence before relying on the claim. latest and baseline proof report coverage metrics: 0 B
  • no-baseline Query p95 Query p95 is currently 0 ms, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 ms
  • no-baseline Portal payload Portal payload is currently 0 B, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 B
  • no-baseline Station features Station features is currently 0, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0
Caveats and gaps
  • Missing signal evidence: Read latency, Station coverage, and Portal payload.
  • No comparable baseline exists for this claim yet.
  • Evidence score is below the adequate threshold.
The proof includes meaningful environmental and route context. The claim has a clear evidence gap. Present signals: none. Missing signals: Bathymetry points, Bathymetry grid, and Mesh routing. There is no comparable baseline yet, so metric values describe current state but cannot prove improvement.
gap 20%
Latest omega-automated-test-run-2026-06-22t19-37-16z | Baseline none | Extra report notes 1
  • missing Bathymetry points Bathymetry points is missing or zero in the latest report. Add this evidence before relying on the claim. latest proof report coverage metrics: 0
  • missing Bathymetry grid Bathymetry grid is missing or zero in the latest report. Add this evidence before relying on the claim. latest proof report coverage metrics: 0
  • missing Mesh routing Mesh routing is missing or zero in the latest report. Add this evidence before relying on the claim. latest proof report coverage metrics: 0
  • no-baseline Bathymetry points Bathymetry points is currently 0, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0
  • no-baseline Mesh routes Mesh routes is currently 0, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0
  • no-baseline Station features Station features is currently 0, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0
Caveats and gaps
  • Missing signal evidence: Bathymetry points, Bathymetry grid, and Mesh routing.
  • No comparable baseline exists for this claim yet.
  • Evidence score is below the adequate threshold.
The communications path is efficient enough for constrained links. The claim has a clear evidence gap. Present signals: none. Missing signals: Wire efficiency, Binary Frame Bytes, and Json Equivalent Bytes. There is no comparable baseline yet, so metric values describe current state but cannot prove improvement.
gap 20%
Latest omega-automated-test-run-2026-06-22t19-37-16z | Baseline none | Extra report notes 1
  • missing Wire efficiency Wire efficiency is missing or zero in the latest report. Add this evidence before relying on the claim. latest and baseline proof report wire metrics: 0 %
  • missing Binary Frame Bytes Binary Frame Bytes is missing or zero in the latest report. Add this evidence before relying on the claim. latest proof report wire metrics: 0
  • missing Json Equivalent Bytes Json Equivalent Bytes is missing or zero in the latest report. Add this evidence before relying on the claim. latest proof report wire metrics: 0
  • no-baseline Wire savings Wire savings is currently 0 %, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 %
  • no-baseline Portal payload Portal payload is currently 0 B, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 B
Caveats and gaps
  • Missing signal evidence: Wire efficiency, Binary Frame Bytes, and Json Equivalent Bytes.
  • No comparable baseline exists for this claim yet.
  • Evidence score is below the adequate threshold.
Storage and performance costs are bounded as data grows. The claim has a clear evidence gap. Present signals: Observation ingest. Missing signals: Storage efficiency and Observation ingest throughput. There is no comparable baseline yet, so metric values describe current state but cannot prove improvement.
gap 40%
Latest omega-automated-test-run-2026-06-22t19-37-16z | Baseline none | Extra report notes 1
  • missing Storage efficiency Storage efficiency is missing or zero in the latest report. Add this evidence before relying on the claim. latest and baseline proof report storage metrics: 0 B/obs
  • present Observation ingest Observation ingest is present at 2. Accepted observations prove the ingest path handled the scenario data. latest proof report ingest metrics: 2
  • missing Observation ingest throughput Observation ingest throughput is missing or zero in the latest report. Add this evidence before relying on the claim. latest and baseline proof report ingest metrics: 0 obs/s
  • no-baseline DB bytes per observation DB bytes per observation is currently 0 B/obs, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 B/obs
  • no-baseline Observation ingest throughput Observation ingest throughput is currently 0 obs/s, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 obs/s
  • no-baseline Write p95 Write p95 is currently 0 ms, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 ms
  • no-baseline Query p95 Query p95 is currently 0 ms, but no comparable baseline exists yet. This is current-state evidence only. latest proof report: 0 ms
Caveats and gaps
  • Missing signal evidence: Storage efficiency and Observation ingest throughput.
  • No comparable baseline exists for this claim yet.
  • Evidence score is below the adequate threshold.

Data-Quality Checks

  • OK Real proof report is available Accepted real reports: 2; quarantined demo/synthetic reports: 12.
  • OK Synthetic/demo reports are quarantined 12 report(s) were excluded from metrics before scoring.
  • GAP Comparable baseline exists Needed to separate real progress from a single isolated run.
  • GAP Multiple proof runs exist Repeated runs help expose noise and regressions.
  • OK More than one scenario is tracked A single scenario can overfit the evidence.
  • GAP Observation sample is large enough Latest run has 2 observations; larger samples make performance/storage claims stronger.
  • GAP Critical claims have adequate evidence Critical claims should not rely on thin evidence.
  • OK No material regressions in latest comparable run Latest comparable run has 0 regressed metrics.
  • GAP AI analyst reviewed current data AI review is advisory and does not inflate the deterministic data-quality score.

QA Matrix

Claims are turned into QA targets with priorities, current evidence strength, regressions, and the next test to run.

PriorityClaimEvidenceRegressionsNext QA Test
P0 The system accepts and stores valid observations correctly.Metrics: write_p95_ms, observation_throughput_per_s thin 60% none Repeat the matching scenario and add the missing signals listed in claim caveats.
P0 Operators can retrieve useful current state quickly.Metrics: query_p95_ms, portal_html_bytes, station_features gap 20% none Repeat the matching scenario and add the missing signals listed in claim caveats.
P1 The proof includes meaningful environmental and route context.Metrics: bathymetry_points, mesh_routes, station_features gap 20% none Repeat the matching scenario and add the missing signals listed in claim caveats.
P1 The communications path is efficient enough for constrained links.Metrics: wire_savings_percent, portal_html_bytes gap 20% none Repeat the matching scenario and add the missing signals listed in claim caveats.
P1 Storage and performance costs are bounded as data grows.Metrics: db_bytes_per_observation, observation_throughput_per_s, write_p95_ms, query_p95_ms gap 40% none Repeat the matching scenario and add the missing signals listed in claim caveats.

Time And Effort Focus

Inferred from proof reports, findings, backlog, and data-quality checks. Direct time-spend tracking requires issue, CI, or work-log imports.

Focus Categories

  • No categoriesNo active findings or backlog categories yet.

Where Time Looks Well Spent

  • SignalUse repeatable proof runs and the measurement plan; those create evidence that compounds over time.

Where Time May Be Wasted

  • SignalEvidence friction: Needed to separate real progress from a single isolated run.
  • SignalEvidence friction: Repeated runs help expose noise and regressions.
  • SignalEvidence friction: Latest run has 2 observations; larger samples make performance/storage claims stronger.
  • SignalEvidence friction: Critical claims should not rely on thin evidence.

Needed For Real Time Accounting

  • SignalIssue status and cycle time
  • SignalCI duration and flake rate
  • SignalManual test or bench-session duration
  • SignalMilestone estimates and actuals

Operating Plan

Create a matching baseline before judging progress.

  • P0
    Strengthen claim: The system accepts and stores valid observations correctly. Current claim evidence is thin and affects trust in the system. Owner: QA/project lead | Impact: high | Confidence: medium Success: claim evidence score Evidence: Repeat the matching scenario and add the missing signals listed in claim caveats. Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status.
  • P0
    Strengthen claim: Operators can retrieve useful current state quickly. Current claim evidence is gap and affects trust in the system. Owner: QA/project lead | Impact: high | Confidence: medium Success: claim evidence score Evidence: Repeat the matching scenario and add the missing signals listed in claim caveats. Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status.
  • P1
    Repeatable Ingest Load Curve Proves the system can accept field observations at useful rates without hidden bottlenecks. Owner: Engineering | Impact: high | Confidence: thin Success: Throughput holds or improves across repeated matching runs with no failed required gates. Evidence: A fresh matching proof report plus before/after metric comparison. Run the same scenario used by the comparable baseline. Capture report.json and import it into BETA. Check whether the target metric improved, stabilized, or regressed again. If it regresses again, profile the owning code path before adding new features.
  • P1
    Endpoint Latency Distribution Operator readiness depends on fast reads, not only successful ingest. Owner: Engineering | Impact: high | Confidence: thin Success: Query p95 is stable or improving, with endpoint-level evidence explaining outliers. Evidence: A fresh matching proof report plus before/after metric comparison. Run the same scenario used by the comparable baseline. Capture report.json and import it into BETA. Check whether the target metric improved, stabilized, or regressed again. If it regresses again, profile the owning code path before adding new features.

Action Tracker

Actions are built from deterministic BETA guidance, manager risks, measurement gaps, and advisory AI output. Statuses are gated by real evidence availability.

beta.action_tracker.v1
Actions 53

tracked recommendations

Active 46

ready to work

Blocked 0

need real data first

Risks 4

manager risks

Avoid 3

guardrails

AI 0

advisory actions

Showing top 12 of 53 tracked actions for this scope.

StatusActionSourceMetricNext StepEvidence Needed
activeP0 Repeat underwater-ops proof for comparison baselineRun the same underwater-ops proof again after the next OMEGA change so BETA can compare against omega-real-underwater-ops-20260606. Project todo ledgerdeterministic project_todo_progressAt least two accepted underwater-ops real reports appear on the OMEGA progress page. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. artifacts/omega-proof/omega-real-underwater-ops-20260606/report.json
activeP0 Strengthen claim: Operators can retrieve useful current state quickly.Current claim evidence is gap and affects trust in the system. Deterministic operating plandeterministic query_p95_ms, portal_html_bytes, station_featuresclaim evidence score Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status. Repeat the matching scenario and add the missing signals listed in claim caveats.
activeP0 Strengthen claim: The system accepts and stores valid observations correctly.Current claim evidence is thin and affects trust in the system. Deterministic operating plandeterministic write_p95_ms, observation_throughput_per_sclaim evidence score Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status. Repeat the matching scenario and add the missing signals listed in claim caveats.
activeP1 Advisory Gate Pass RatePercent of accepted OMEGA proof reports where advisory performance checks pass Measurement plandeterministic advisory_gate_pass_rateThe metric has a real baseline, a repeat run, and a clear decision rule. Record this value in a real report, metrics import, bench log, CI result, or field note. The metric has a real baseline, a repeat run, and a clear decision rule.
activeP1 Advisory Gate Pass RateThis planned metric is not active yet. Manager metric backlogdeterministic advisory_gate_pass_rateThe metric has a real baseline, a repeat run, and a clear decision rule. Add this metric to a real report, CI import, bench log, field note, or manual evidence record. The metric has a real baseline, a repeat run, and a clear decision rule.
activeP1 Capture hardware and field validation evidenceAdd real board, radio, power, thermal, OLED, GNSS, or dockside/field evidence when available. Project todo ledgerdeterministic project_todo_progressAt least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. At least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report.
activeP1 Connect AI AnalystOllama reviews weak evidence, missing measurements, experiment ideas, and next actions. Missing data sourcedeterministic aisource connected Use auto-strong for serious reviews; keep deterministic metrics as the source of truth. Use auto-strong for serious reviews; keep deterministic metrics as the source of truth.
activeP1 Connect Comparable BaselineA matching prior scenario separates real progress from one-off numbers. Missing data sourcedeterministic evidencesource connected Repeat the same scenario after changes so every metric has a before/after comparison. Repeat the same scenario after changes so every metric has a before/after comparison.
activeP1 Connect Field And Bench LogsPower, thermal, calibration, endurance, and human acceptance data prove real-world readiness. Missing data sourcedeterministic fieldsource connected Create a simple CSV or report.json path for bench and field validation evidence. Create a simple CSV or report.json path for bench and field validation evidence.
activeP1 Connect Issue And CI HistoryProject-management evidence needs defects, milestones, build pass rate, coverage, and flaky-test history. Missing data sourcedeterministic managementsource connected Add a connector/importer for issues, milestones, CI status, coverage, and release notes. Add a connector/importer for issues, milestones, CI status, coverage, and release notes.
activeP1 Connect OMEGA CI and test historyAttach current OMEGA test/proof workflow outputs, CI runs, and relevant pytest results as BETA sources. Project todo ledgerdeterministic project_todo_progressOMEGA source coverage shows CI/test history as connected and manager risks stop asking for it. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. OMEGA source coverage shows CI/test history as connected and manager risks stop asking for it.
activeP1 Connect Quarantined Demo ReportsSynthetic, demo, smoke, fixture, and local proof reports stay visible for audit but are excluded from metrics. Missing data sourcedeterministic evidencesource connected Replace quarantined reports with real evidence or mark real reports explicitly with metadata.data_authenticity=real. Replace quarantined reports with real evidence or mark real reports explicitly with metadata.data_authenticity=real.

AI can suggest actions, but BETA only treats metrics, imported inputs, proof reports, and work logs as evidence.

Metric Intelligence

Each metric now has risk, confidence, volatility, streaks, and a recommended action.

MetricRiskLatestTrendVolatilityRecommended Action
Advisory Gate Pass RatePercent of accepted OMEGA proof reports where advisory performance checks pass stablethin confidence 0 %best 0 % no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Bathymetry pointsShows that terrain/depth data is included in the proof evidence. stablethin confidence 0best 0 no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
DB bytes per observationShows storage efficiency and whether growth is becoming expensive. stablethin confidence 0 B/obsbest 0 B/obs no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Mesh routesShows whether route/network behavior is being exercised. stablethin confidence 0best 0 no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Observation ingest throughputMeasures gateway ingest speed on the proof scenario stablethin confidence 0 obs/sbest 0 obs/s no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Portal payloadLarge payloads hurt slow-link and tunnel usability. stablethin confidence 0 Bbest 0 B no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Query p95Measures portal/API query responsiveness under the proof scenario stablethin confidence 0 msbest 0 ms no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Required Gate Pass RatePercent of accepted OMEGA proof reports where required checks pass stablethin confidence 0 %best 0 % no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Station featuresShows that station coverage is present rather than only raw observations. stablethin confidence 0best 0 no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Test FailuresFailing OMEGA tests that need fixing stablethin confidence 0 countbest 0 count no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Unit Test Pass RatePercent of OMEGA pytest tests passing stablethin confidence 100.0 %best 100.0 % no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.
Wire savingsMeasures binary mesh protocol efficiency versus JSON stablethin confidence 0 %best 0 % no-baselinestreak R0 / I0 0 %gap 0 % Keep this in the regression suite while focusing on weaker metrics.

Project Manager

Create a matching baseline before judging progress.

Posture build

manager mode

Readiness 39.80 %

needs setup

Source Coverage 46.70 %

connected sources

Projects 1

tracked builds

Priorities 5

current actions

Risks 4

tracked manager risks

Data Gaps 8

sources to connect

Metrics Backlog 8

planned metrics

Inputs 15

project evidence files

Open Todos 3

committed work

Blocked Todos 0

plan blockers

Todo Progress 0 %

done excluding dropped

Work Logs 4

effort records

Workflow thin

health label

Project Todo Ledger

3 todo item(s) are tracked across 1 project(s): 3 open, 0 blocked, 0 done.

Todos 3

tracked commitments

Open 3

todo, doing, blocked

Doing 1

current focus

Blocked 0

needs decision

Overdue 0

past due

Done 0

completed

Progress 0 %

done excluding dropped

Active Todo Board

  • P0
    Repeat underwater-ops proof for comparison baseline OMEGA | doing | due none | owner OMEGA/BETA ID: repeat-underwater-ops-proof-for-comparison-baseline-2026-06-06t190938z Area: evidence Success: At least two accepted underwater-ops real reports appear on the OMEGA progress page. Blocker: none Evidence: artifacts/omega-proof/omega-real-underwater-ops-20260606/report.json
  • P1
    Capture hardware and field validation evidence OMEGA | todo | due none | owner OMEGA ID: capture-hardware-and-field-validation-evidence-2026-06-06t190938z Area: field validation Success: At least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report. Blocker: none Evidence: none
  • P1
    Connect OMEGA CI and test history OMEGA | todo | due none | owner OMEGA ID: connect-omega-ci-and-test-history-2026-06-06t190938z Area: source coverage Success: OMEGA source coverage shows CI/test history as connected and manager risks stop asking for it. Blocker: none Evidence: none

Recent Todo Changes

  • P1
    Capture hardware and field validation evidence OMEGA | todo | due none | owner OMEGA ID: capture-hardware-and-field-validation-evidence-2026-06-06t190938z Area: field validation Success: At least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report. Blocker: none Evidence: none
  • P1
    Connect OMEGA CI and test history OMEGA | todo | due none | owner OMEGA ID: connect-omega-ci-and-test-history-2026-06-06t190938z Area: source coverage Success: OMEGA source coverage shows CI/test history as connected and manager risks stop asking for it. Blocker: none Evidence: none
  • P0
    Repeat underwater-ops proof for comparison baseline OMEGA | doing | due none | owner OMEGA/BETA ID: repeat-underwater-ops-proof-for-comparison-baseline-2026-06-06t190938z Area: evidence Success: At least two accepted underwater-ops real reports appear on the OMEGA progress page. Blocker: none Evidence: artifacts/omega-proof/omega-real-underwater-ops-20260606/report.json

Action Tracker

Actions are built from deterministic BETA guidance, manager risks, measurement gaps, and advisory AI output. Statuses are gated by real evidence availability.

beta.action_tracker.v1
Actions 53

tracked recommendations

Active 46

ready to work

Blocked 0

need real data first

Risks 4

manager risks

Avoid 3

guardrails

AI 0

advisory actions

Showing top 12 of 53 tracked actions for this scope.

StatusActionSourceMetricNext StepEvidence Needed
activeP0 Repeat underwater-ops proof for comparison baselineRun the same underwater-ops proof again after the next OMEGA change so BETA can compare against omega-real-underwater-ops-20260606. Project todo ledgerdeterministic project_todo_progressAt least two accepted underwater-ops real reports appear on the OMEGA progress page. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. artifacts/omega-proof/omega-real-underwater-ops-20260606/report.json
activeP0 Strengthen claim: Operators can retrieve useful current state quickly.Current claim evidence is gap and affects trust in the system. Deterministic operating plandeterministic query_p95_ms, portal_html_bytes, station_featuresclaim evidence score Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status. Repeat the matching scenario and add the missing signals listed in claim caveats.
activeP0 Strengthen claim: The system accepts and stores valid observations correctly.Current claim evidence is thin and affects trust in the system. Deterministic operating plandeterministic write_p95_ms, observation_throughput_per_sclaim evidence score Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status. Repeat the matching scenario and add the missing signals listed in claim caveats.
activeP1 Advisory Gate Pass RatePercent of accepted OMEGA proof reports where advisory performance checks pass Measurement plandeterministic advisory_gate_pass_rateThe metric has a real baseline, a repeat run, and a clear decision rule. Record this value in a real report, metrics import, bench log, CI result, or field note. The metric has a real baseline, a repeat run, and a clear decision rule.
activeP1 Advisory Gate Pass RateThis planned metric is not active yet. Manager metric backlogdeterministic advisory_gate_pass_rateThe metric has a real baseline, a repeat run, and a clear decision rule. Add this metric to a real report, CI import, bench log, field note, or manual evidence record. The metric has a real baseline, a repeat run, and a clear decision rule.
activeP1 Capture hardware and field validation evidenceAdd real board, radio, power, thermal, OLED, GNSS, or dockside/field evidence when available. Project todo ledgerdeterministic project_todo_progressAt least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. At least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report.
activeP1 Connect AI AnalystOllama reviews weak evidence, missing measurements, experiment ideas, and next actions. Missing data sourcedeterministic aisource connected Use auto-strong for serious reviews; keep deterministic metrics as the source of truth. Use auto-strong for serious reviews; keep deterministic metrics as the source of truth.
activeP1 Connect Comparable BaselineA matching prior scenario separates real progress from one-off numbers. Missing data sourcedeterministic evidencesource connected Repeat the same scenario after changes so every metric has a before/after comparison. Repeat the same scenario after changes so every metric has a before/after comparison.
activeP1 Connect Field And Bench LogsPower, thermal, calibration, endurance, and human acceptance data prove real-world readiness. Missing data sourcedeterministic fieldsource connected Create a simple CSV or report.json path for bench and field validation evidence. Create a simple CSV or report.json path for bench and field validation evidence.
activeP1 Connect Issue And CI HistoryProject-management evidence needs defects, milestones, build pass rate, coverage, and flaky-test history. Missing data sourcedeterministic managementsource connected Add a connector/importer for issues, milestones, CI status, coverage, and release notes. Add a connector/importer for issues, milestones, CI status, coverage, and release notes.
activeP1 Connect OMEGA CI and test historyAttach current OMEGA test/proof workflow outputs, CI runs, and relevant pytest results as BETA sources. Project todo ledgerdeterministic project_todo_progressOMEGA source coverage shows CI/test history as connected and manager risks stop asking for it. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. OMEGA source coverage shows CI/test history as connected and manager risks stop asking for it.
activeP1 Connect Quarantined Demo ReportsSynthetic, demo, smoke, fixture, and local proof reports stay visible for audit but are excluded from metrics. Missing data sourcedeterministic evidencesource connected Replace quarantined reports with real evidence or mark real reports explicitly with metadata.data_authenticity=real. Replace quarantined reports with real evidence or mark real reports explicitly with metadata.data_authenticity=real.

AI can suggest actions, but BETA only treats metrics, imported inputs, proof reports, and work logs as evidence.

Operating Metrics

  • Project profiles: 1 Each tracked build needs a setup profile, goals, source paths, and project type.
  • Project goals: 3 Goals tell BETA what outcomes the metrics are supposed to support.
  • Connected data sources: 7 / 15 Connected sources make the manager brief factual instead of guessy.
  • Project inputs: 8 Issues, CI, AI sessions, docs, bench logs, and field logs explain why metrics moved.
  • Local repo snapshots: 1 Snapshots show Git state, source/test/doc balance, CI presence, and project structure.
  • Snapshot source files: 128 Source volume helps size the project and compare testing/documentation balance.
  • Snapshot test files: 52 Test volume is an early signal for regression protection and QA maturity.
  • Dirty repos: 1 Dirty worktrees can make evidence hard to reproduce unless changes are explained.

Tracked Projects

omegaplanned
OMEGA

mixed | evidence_history, firmware, software

Goal: Prove OMEGA gateway proof runs pass required and advisory gates on real measured pipeline data Files: 424 of 424 Inputs: 8 (ai-session: 3, docs: 1, metrics: 2, repo: 2) Todos: 3 open, 0 blocked, 0 % done AI: llama3.2:latest

Current Manager Priorities

  • P0
    Strengthen claim: The system accepts and stores valid observations correctly. Current claim evidence is thin and affects trust in the system. Owner: QA/project lead | Impact: high | Confidence: medium Success: claim evidence score Evidence: Repeat the matching scenario and add the missing signals listed in claim caveats. Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status.
  • P0
    Strengthen claim: Operators can retrieve useful current state quickly. Current claim evidence is gap and affects trust in the system. Owner: QA/project lead | Impact: high | Confidence: medium Success: claim evidence score Evidence: Repeat the matching scenario and add the missing signals listed in claim caveats. Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status.
  • P1
    Repeatable Ingest Load Curve Proves the system can accept field observations at useful rates without hidden bottlenecks. Owner: Engineering | Impact: high | Confidence: thin Success: Throughput holds or improves across repeated matching runs with no failed required gates. Evidence: A fresh matching proof report plus before/after metric comparison. Run the same scenario used by the comparable baseline. Capture report.json and import it into BETA. Check whether the target metric improved, stabilized, or regressed again. If it regresses again, profile the owning code path before adding new features.
  • P1
    Endpoint Latency Distribution Operator readiness depends on fast reads, not only successful ingest. Owner: Engineering | Impact: high | Confidence: thin Success: Query p95 is stable or improving, with endpoint-level evidence explaining outliers. Evidence: A fresh matching proof report plus before/after metric comparison. Run the same scenario used by the comparable baseline. Capture report.json and import it into BETA. Check whether the target metric improved, stabilized, or regressed again. If it regresses again, profile the owning code path before adding new features.
  • P2
    Owner: Project manager | Impact: | Confidence: Success: Evidence:

Manager Risks

  • Missing AI Analyst Ollama reviews weak evidence, missing measurements, experiment ideas, and next actions. Mitigation: Use auto-strong for serious reviews; keep deterministic metrics as the source of truth. Owner: Project manager
  • Missing Field And Bench Logs Power, thermal, calibration, endurance, and human acceptance data prove real-world readiness. Mitigation: Create a simple CSV or report.json path for bench and field validation evidence. Owner: Project manager
  • Missing Issue And CI History Project-management evidence needs defects, milestones, build pass rate, coverage, and flaky-test history. Mitigation: Add a connector/importer for issues, milestones, CI status, coverage, and release notes. Owner: Project manager
  • Blocked or rework time is high The work ledger shows 30.0% of tracked effort as blocked, rework, or unclear. Mitigation: Review the top work-log waste signals and remove one blocker before opening new scope. Owner: Project manager

Project Controls

  • Project intake active 1 project profile(s) Keep project goals, paths, type, and claim list current.
  • Evidence intake active 8 ingested project input(s) Import the source that explains the latest work or blocker.
  • Measurement backlog active 16 planned measurement(s) Attach every major claim to a repeatable metric and test method.
  • Snapshot intelligence active 1 snapshot(s), 128 source file(s), 52 test file(s) Collect snapshots after meaningful repo changes and review dirty repo, CI, test, and doc signals.
  • Todo ledger active 3 todo item(s), 3 open, 0 blocked Keep active todos current and close done work with evidence.
  • Work ledger active 4 logged work session(s) Record minutes, category, outcome, evidence, blocker, and next step after meaningful work.
  • AI collaboration active 3 AI session input(s) Capture AI decisions, discarded ideas, and tested recommendations.

Project Setup Wizard

Make the project measurable by defining goals, claims, metrics, source inputs, and the evidence packet BETA should expect.

omega

Collect Project Snapshot

Reads configured paths and captures Git state, file mix, test, docs, config, and CI signals.

Run Tests

Runs the project's configured test command and records pass rate, coverage, and failures as real CI evidence.

Connect A Source

Create Evidence Template

Source Connection Plan

Connected 4

source connectors

Planned 3

still needs data

Coverage 57.10 %

source plan

ai-sessionconnected
AI Work Sessions

Tracks what AI suggested, what was tried, and whether later metrics improved.

Metric: ai_recommendation_follow_through, accepted_ai_actions No path connected yet. Next: Save useful AI summaries with recommendation, action, evidence, and result fields.
docsconnected
Requirements And Design Docs

Connects project goals, claims, acceptance criteria, and design decisions to evidence.

Metric: claim_coverage, acceptance_criteria_coverage No path connected yet. Next: Attach requirements, design notes, test matrices, decision logs, and acceptance criteria.
metricsconnected
External Metrics

Imports existing numeric project metrics that BETA should trend and reason about.

Metric: project_custom_metric_count No path connected yet. Next: Attach CSV, JSON, or report files with stable metric names and timestamps.
repoconnected
Local Project Snapshot

Tracks repository state, file mix, test/doc/config signals, CI hints, and dirty worktree risk.

Metric: source_file_count, test_file_count, doc_file_count, dirty_repo_count C:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave Next: Use Collect Project Snapshot after important work so BETA can compare source, test, docs, and Git-state changes.
benchplanned
Bench Evidence

Tracks measured setup, hardware, performance, calibration, and validation runs.

Metric: bench_pass_rate, measured_failure_count, setup_time_minutes No path connected yet. Next: Attach bench CSVs, checklists, calibration logs, photos, or measured report.json files.
ciplanned
CI And Test History

Tracks build health, test pass rate, coverage, and failed checks.

Metric: unit_test_pass_rate, code_coverage_percent, test_failure_count No path connected yet. Next: Use Import Test Results (beta import-tests) on a JUnit XML, pytest JSON, coverage report, or CI log after each meaningful build.
issuesplanned
Issue And Milestone History

Tracks planned work, stale work, blockers, cycle time, and release scope.

Metric: open_issue_count, blocked_issue_count, cycle_time_days No path connected yet. Next: Export GitHub/Jira issues or keep a simple CSV of issue status and milestone dates.

Project Goals & Setup

Use this setup guide to turn BETA from a dashboard into a project manager: goals define intent, metrics define proof, and evidence changes the plan.

mixed
Project OMEGA omega
Project file C:\Users\jdcap\Documents\Projects\BETA\.beta\projects\omega\project.json Customize goals, claims, metrics, paths, and evidence sources here.

Setup And Customization Commands

  • 1
    Create a separated project Creates a project page, project.json, scan profile, verification plan, and project-scoped dashboard data. .\dev.ps1 init-project -ProjectPath C:\path\to\build -ProjectName "Bench Prototype" -ProjectType hardware -Goal "Prove stable bench operation"
  • 2
    Add goals, claims, or custom metrics Custom goals and metrics become planning inputs, manager context, and measurement backlog. .\dev.ps1 configure-project -ProjectKey omega -Goal "Make field setup repeatable" -Metric "setup_success_rate|%|Tracks whether setup succeeds without manual rescue" -Claim "Operators can identify a node and trust its status"
  • 3
    Connect real project context Issues, CI, bench logs, field notes, docs, and AI sessions explain why a metric moved. .\dev.ps1 ingest-info -ProjectKey omega -InfoPath C:\path\to\issues-or-notes.csv -SourceType issues -Note "current backlog"
  • 4
    Record work and evidence Work logs tell the manager where time is productive, blocked, rework, or evidence-producing. .\dev.ps1 record-work -ProjectKey omega -WorkTitle "Validation pass" -WorkMinutes 45 -WorkStatus tested -WorkEvidence "report.json"
  • 5
    Run the manager loop Refreshes deterministic analysis, then asks the local AI to turn it into actionable project guidance. .\dev.ps1 refresh; .\dev.ps1 manage-ai -Model gemma4:latest

What You Can Customize

  • goals: 3 What the project is trying to improve or prove.
  • custom_claims: 3 Statements BETA should try to connect to evidence.
  • custom_metrics: 7 Project-specific measurements that should appear in the planning backlog.
  • evidence_sources: 9 Where useful proof can come from: CI, bench, field, docs, issues, AI sessions.
  • test_tools: 6 Tools or workflows that can produce proof.
  • paths: 1 Repo, hardware folder, docs folder, or build path BETA should scan.

Planning Loop

  • SignalGoal: decide what outcome matters.
  • SignalClaim: write the thing you want to be able to say is true.
  • SignalMetric: define the number, pass/fail, or evidence signal that would prove it.
  • SignalScenario: run the same test or validation path repeatedly.
  • SignalEvidence: import the report, CI result, bench log, field note, or operator signoff.
  • SignalManager decision: focus, stop, protect, or connect a missing source.

Configured Goals

  • SignalProve OMEGA gateway proof runs pass required and advisory gates on real measured pipeline data
  • SignalTrack software, firmware, mesh, portal, and field-readiness progress from accepted evidence only
  • SignalUse BETA to identify the next highest-value OMEGA test, source input, and blocker

Custom Claims

  • SignalOMEGA can ingest simulated operational data through the real gateway pipeline and expose useful map, mesh, bathymetry, acoustic, and portal outputs.
  • SignalOMEGA proof reports are repeatable enough for BETA to compare progress over time.
  • SignalOMEGA firmware and gateway changes can be connected to project goals, todos, and evidence instead of loose notes.

Custom Metrics

  • P1
    Required Gate Pass Rate required_gate_pass_rate (%) Percent of accepted OMEGA proof reports where required checks pass
  • P1
    Advisory Gate Pass Rate advisory_gate_pass_rate (%) Percent of accepted OMEGA proof reports where advisory performance checks pass
  • P1
    Observation Throughput observation_throughput_per_s (obs/s) Measures gateway ingest speed on the proof scenario
  • P1
    Query P95 query_p95_ms (ms) Measures portal/API query responsiveness under the proof scenario
  • P1
    Wire Savings wire_savings_percent (%) Measures binary mesh protocol efficiency versus JSON
  • P1
    Unit Test Pass Rate unit_test_pass_rate (%) Percent of OMEGA pytest tests passing
  • P1
    Test Failures test_failure_count (count) Failing OMEGA tests that need fixing

Starter Todo Cycle

Use this to create the first measurable planning loop: real proof, connected project history, and field or bench validation.

connected
P0Import one accepted real proof report

Run or import one real bench, field, CI, hardware, or project proof report with no demo, synthetic, or local-proof markers.

Done when: Accepted real report count is greater than zero and appears in the project evidence page.
P1Connect issue and CI history

Import issues, milestones, build status, test pass rate, coverage, flaky-test notes, and release notes.

Done when: Issue and CI sources are connected and visible in source coverage.
P1Capture field and bench logs

Add bench or field CSV, log, checklist, or report paths for power, thermal, calibration, endurance, and operator notes.

Done when: Field and bench sources are connected with at least one evidence-backed work or proof record.

Todo And Work Control

Add committed work, update blockers, and log how time was spent so the manager view can explain progress and waste.

omega

Add Todo

Update Todo

Record Work Session

Project Todo Ledger

3 todo item(s) are tracked across 1 project(s): 3 open, 0 blocked, 0 done.

Todos 3

tracked commitments

Open 3

todo, doing, blocked

Doing 1

current focus

Blocked 0

needs decision

Overdue 0

past due

Done 0

completed

Progress 0 %

done excluding dropped

Active Todo Board

  • P0
    Repeat underwater-ops proof for comparison baseline OMEGA | doing | due none | owner OMEGA/BETA ID: repeat-underwater-ops-proof-for-comparison-baseline-2026-06-06t190938z Area: evidence Success: At least two accepted underwater-ops real reports appear on the OMEGA progress page. Blocker: none Evidence: artifacts/omega-proof/omega-real-underwater-ops-20260606/report.json
  • P1
    Capture hardware and field validation evidence OMEGA | todo | due none | owner OMEGA ID: capture-hardware-and-field-validation-evidence-2026-06-06t190938z Area: field validation Success: At least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report. Blocker: none Evidence: none
  • P1
    Connect OMEGA CI and test history OMEGA | todo | due none | owner OMEGA ID: connect-omega-ci-and-test-history-2026-06-06t190938z Area: source coverage Success: OMEGA source coverage shows CI/test history as connected and manager risks stop asking for it. Blocker: none Evidence: none

Recent Todo Changes

  • P1
    Capture hardware and field validation evidence OMEGA | todo | due none | owner OMEGA ID: capture-hardware-and-field-validation-evidence-2026-06-06t190938z Area: field validation Success: At least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report. Blocker: none Evidence: none
  • P1
    Connect OMEGA CI and test history OMEGA | todo | due none | owner OMEGA ID: connect-omega-ci-and-test-history-2026-06-06t190938z Area: source coverage Success: OMEGA source coverage shows CI/test history as connected and manager risks stop asking for it. Blocker: none Evidence: none
  • P0
    Repeat underwater-ops proof for comparison baseline OMEGA | doing | due none | owner OMEGA/BETA ID: repeat-underwater-ops-proof-for-comparison-baseline-2026-06-06t190938z Area: evidence Success: At least two accepted underwater-ops real reports appear on the OMEGA progress page. Blocker: none Evidence: artifacts/omega-proof/omega-real-underwater-ops-20260606/report.json

Work Session Ledger

4 work session(s) are logged across 1 project(s), totaling 0.83 tracked hours.

Sessions 4

logged work blocks

Tracked Time 0.83 h

total effort

Productive 70.00 %

completed, shipped, tested, evidence, decided

Evidence Work 100.0 %

tied to proof

Blocked/Rework 30.00 %

friction signal

AI-Assisted 0.58 h

tracked AI use

Where Time Is Going

CategoryTracked Hours
testing 0.42 h
evidence 0.42 h

Recent Work Sessions

  • blocked
    Test run timed out: pytest-full OMEGA | testing | 15 min | 2026-06-22T20:06:33Z Outcome: Test command exceeded the 900s timeout. Evidence: none recorded Next: Increase --timeout-s or investigate a hanging test.
  • tested
    Ran tests: pytest-smoke OMEGA | testing | 0 min | 2026-06-22T19:37:16Z Outcome: 2/2 passed, 0 failed; exit 0 Evidence: C:\Users\jdcap\Documents\Projects\BETA\.beta\projects\omega\evidence\omega-automated-test-run-2026-06-22t19-37-16z\report.json Next: none recorded
  • tested
    Refresh OMEGA to upstream and generate real proof OMEGA | evidence | 25 min | 2026-06-06T19:09:57Z Outcome: Fast-forwarded OMEGA to origin/main 895b12d, installed declared dev dependencies, ran gateway.proof underwater-ops, and imported the accepted report into BETA. Evidence: OMEGA report: C:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-wave\artifacts\omega-proof\omega-real-underwater-ops-20260606\report.json; BETA page shows real_count=1. Next: Repeat underwater-ops after next OMEGA change to create a comparable baseline.
  • tested
    Review upstream firmware changes OMEGA | testing | 10 min | 2026-06-02T05:15:24Z Outcome: Pulled upstream to d54cf42, reviewed firmware identity and OLED telemetry changes, ran proof-runner test, and refreshed the BETA OMEGA project scan. Evidence: git pull 5852114..d54cf42; pytest tests/test_omega_proof.py: 1 passed; BETA plan scan: 374 files Next: Add firmware identity, battery, temperature, OLED, and physical-board verification to the OMEGA evidence capture plan.

Deterministic Findings

  • No open findingsThe latest run has no deterministic findings.

Improvement Backlog

  • ClearNo generated actions for the latest run.

Data Version & Model Provenance

This records exactly what generated this page, what schemas were used, and whether an Ollama model contributed advisory analysis.

beta-2026-06-22t21-...
Data version beta-2026-06-22t21-24-40z-228d8bfa07 228d8bfa0747582e20b5a087c9598f33458e30e45e51ca2e9ca24ea70864074b
BETA app 0.2.0 Build Environment for Testing & Analytics
Analysis schema beta.analysis.v1 BETA deterministic engine
Generated 2026-06-22T21:24:40Z C:\Users\jdcap\Documents\Projects\BETA\.beta
AI model none AI not used; ollama model none; available=False; usable=False; source=none
Data policy deterministic source of truth Only accepted real reports are used for metrics, graphs, claims, progress, and AI analysis. Synthetic/demo/local proof reports are quarantined.
Real reports 2 Accepted reports used for metrics, graphs, claims, and progress.
Quarantined reports 12 Visible for audit but excluded from proof calculations.

Project Version Records

  • OMEGA Config: v3 (explicit) | Schema: beta.project.v1 Project AI plan: llama3.2:latest | Profile: 2026-06-06T19:08:02Z | Plan: 2026-06-06T19:08:02Z

Version Change Since Previous Snapshot

A new data version was generated, but tracked evidence/model/project-config fields did not materially change.

snapshot-only
Current data version beta-2026-06-22t21-24-40z-228d8bfa07 2026-06-22T21:24:40Z
Previous data version beta-2026-06-22t20-02-03z-2fe016a499 2026-06-22T20:02:03Z
  • No material tracked changesThis snapshot did not change the tracked evidence/model/project-config posture.

Data Version History

Each row is a generated dashboard/report snapshot for this scope. Use it to see which BETA version, model, and project config produced past data.

12 shown
Data VersionRecordedBETAAI ModelRealQuarantinedProject Configs
beta-2026-06-22t21-24-4...project:omega 2026-06-22T21:24:40Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t20-02-0...project:omega 2026-06-22T20:02:03Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-52-5...project:omega 2026-06-22T19:52:58Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-51-1...project:omega 2026-06-22T19:51:17Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-51-1...project:omega 2026-06-22T19:51:16Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-41-0...project:omega 2026-06-22T19:41:01Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-41-0...project:omega 2026-06-22T19:41:01Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-37-1...project:omega 2026-06-22T19:37:17Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-37-1...project:omega 2026-06-22T19:37:16Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-37-0...project:omega 2026-06-22T19:37:01Z 0.2.0 none 1 12 omega v3 (explicit)
beta-2026-06-22t19-32-0...project:omega 2026-06-22T19:32:09Z 0.2.0 none 1 12 omega v2 (explicit)
beta-2026-06-22t19-32-0...project:omega 2026-06-22T19:32:08Z 0.2.0 none 1 12 omega v2 (explicit)
baseline

Create a comparable baseline before judging progress.

The dashboard needs a matching prior scenario before improvement claims are meaningful.

Data quality 38% thin

Focus Metrics

  • No metric regressionsKeep collecting repeated runs and broader scenarios.

Next Moves

  • Repeatable Ingest Load Curve P1 | Backend/performance
  • Endpoint Latency Distribution P1 | Gateway/API
  • Storage Growth Curve P1 | Data/storage
  • Firmware Boot And I/O Trace P1 | Firmware

Missing Inputs

  • Repeatability Run the same scenario several times before treating a change as proven.
  • Sample Scale Run proof scenarios at 100, 500, and 1000+ observations and compare the curves.
  • Source Connection Plan Register planned sources, then connect real files or folders as they become available.
  • Comparable Baseline Repeat the same scenario after changes so every metric has a before/after comparison.
Connected 7

evidence inputs

Partial 3

needs stronger proof

Missing 4

not imported yet

Active 6

tracked measurements

Snapshot Intelligence

Local project-state data from the configured path. This supports planning and QA; it does not replace proof reports.

strong
Snapshot Score 85.00 %

2026-06-22T19:37:01Z

Source Files 128

scanned source

Test Files 52

ratio 0.406

Docs 61

ratio 0.477

CI Workflows 1

detected

Dirty Repos 1

needs explanation

Trends Over Time

Collect at least two snapshots to chart source-tree and Git-state trends.

PathFilesSource/Test/DocsGitDirty
C:\Users\jdcap\OneDrive\Documents\OMEGA Proof\OMEGA-waveexists 437 128/52/61 main 0f5da76 4

Recommended Improvements

  • Resolve or explain dirty repo state1 scanned repo(s) have uncommitted or untracked changes. Next: Commit, stash, or record a work session explaining the active changes before treating metrics as stable.

What BETA Noticed

  • Snapshot noteGit worktree has uncommitted or untracked changes.

Real Evidence Capture Kit

OMEGA has real evidence; repeat matching scenarios to prove improvement.

collect-repeat-baseline

Starter Scenarios

benchP1
Bench Validation Baseline

Creates the first accepted real baseline so charts and claims stop being empty.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId bench-validation -CollectionType bench .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId bench-validation -CollectionType bench -RequiredPassed -AdvisoryPassed -ObservationsAccepted 100 One measured report imports as real, required gates pass, and the dashboard shows one real run.
fieldP1
Field-Link Load Check

Connects portal/API responsiveness to real network conditions instead of local-only timing.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId field-link-load -CollectionType field .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId field-link-load -CollectionType field -RequiredPassed -ObservationsAccepted 100 -PortalHtmlBytes 1 -QueryP95Ms 1 Portal payload and query p95 are recorded from a constrained or remote path.
ciP1
CI Regression Evidence

Adds build/test pass history so project progress is not inferred only from proof runs.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId ci-regression -CollectionType ci .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId ci-regression -CollectionType ci -RequiredPassed -ObservationsAccepted 1 CI result evidence is attached and repeat failures trend down over time.
firmwareP1
Firmware Boot And I/O Trace

Firmware projects need boot and sensor/I/O evidence before readiness claims are trusted.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId firmware-boot-io -CollectionType firmware .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId firmware-boot-io -CollectionType firmware -RequiredPassed -ObservationsAccepted 1 Serial boot logs or firmware checks are summarized in the report evidence notes.
firmwareP1
Firmware Identity, Health, And OLED Evidence

Connects the new OMEGA firmware identity, board-lineage, OLED, and health telemetry fields to real evidence instead of code-only claims.

.\dev.ps1 evidence-template -ProjectKey omega -ScenarioId firmware-identity-health -CollectionType firmware .\dev.ps1 record-evidence -ProjectKey omega -ScenarioId firmware-identity-health -CollectionType firmware -RequiredPassed -ObservationsAccepted 1 -Note "Capture serial heartbeat and OLED evidence for operator_label physical_board_tag physical_mac esp_temp_c battery fields" Serial/JSON/OLED evidence shows operator_label, physical_board_tag, physical_mac, mesh/node identity, esp_temp_c, battery/PMU health where supported, and readable OLED state.

Required Report Fields

  • metadata.data_authenticityrealLets BETA accept the report as usable evidence instead of quarantining it.
  • metadata.collection_typebench | field | ci | firmware | hardware | operator | measuredTells BETA what kind of real-world source produced the measurement.
  • scenario.scenario_idstable scenario idMatching scenario ids allow before/after comparisons over time.
  • overall.required_passedtrue/falseCorrectness gates are the minimum proof before performance claims matter.
  • metrics.ingest.counts.observations.acceptedintegerSample size affects confidence in throughput, storage, and latency claims.
  • analysis.evidence[].summaryshort measured source noteKeeps the metric tied to the actual test, bench note, CI run, or field observation.
  • analysis.evidence[].sourcelog/photo/serial/CI/bench source referenceLets a reviewer trace the metric back to the file, capture, or note that produced it.

Metric Field Map

MeasurementReport FieldPriority
Repeatable Ingest Load Curveobservation_throughput_per_s metrics.ingest.observation_throughput_per_s P1
Endpoint Latency Distributionquery_p95_ms metrics.queries.all_queries_ms.p95_ms P1
Storage Growth Curvedb_bytes_per_observation metrics.efficiency.db_bytes_per_observation P1
Field-Link Portal Loadportal_html_bytes metrics.coverage.portal_html_bytes P2
Reliability And Failure Historyfailure_rate analysis.evidence[] or imported CI input P2
Human Acceptance Signoffoperator_acceptance analysis.evidence[] or field/operator signoff P2
Firmware Boot And I/O Tracefirmware_boot_success_rate analysis.evidence[] or firmware report P1
Firmware Identity And Board Lineagefirmware_identity_fields_present analysis.evidence[] or firmware JSON/serial frame P1
Firmware Power And Thermal Healthfirmware_health_telemetry_present analysis.evidence[] or firmware JSON/serial frame P1
Operator OLED Readabilityoperator_oled_readability analysis.evidence[] or operator/OLED inspection note P2
Required Gate Pass Raterequired_gate_pass_rate metrics.custom.required_gate_pass_rate P1
Advisory Gate Pass Rateadvisory_gate_pass_rate metrics.custom.advisory_gate_pass_rate P1
Wire Savingswire_savings_percent metrics.wire.savings_percent P1
Unit Test Pass Rateunit_test_pass_rate metrics.custom.unit_test_pass_rate P1
Test Failurestest_failure_count metrics.custom.test_failure_count P1
Test Pass Ratetest_pass_rate analysis.evidence[] or imported CI input P2

Data Sources

These inputs determine whether the evidence is real, repeatable, and useful for project decisions.

evidenceconnected
Real Proof Reports

Only accepted real run reports drive charts, regressions, findings, claims, and AI analysis.

Signal: 2 Next: Import real bench, field, CI, hardware, or project proof reports with no demo/synthetic/local-proof markers.
evidencequarantined
Quarantined Demo Reports

Synthetic, demo, smoke, fixture, and local proof reports stay visible for audit but are excluded from metrics.

Signal: 12 Next: Replace quarantined reports with real evidence or mark real reports explicitly with metadata.data_authenticity=real.
evidencemissing
Comparable Baseline

A matching prior scenario separates real progress from one-off numbers.

Signal: none Next: Repeat the same scenario after changes so every metric has a before/after comparison.
evidencepartial
Repeatability

Multiple runs expose noise, flaky behavior, and repeated regressions.

Signal: 2 Next: Run the same scenario several times before treating a change as proven.
evidenceconnected
Scenario Diversity

More scenarios reduce the risk of proving only one narrow demo path.

Signal: 2 Next: Add at least one scale or field-like scenario beside the current coastal proof.
evidencepartial
Sample Scale

Larger samples make performance, storage, and reliability claims harder to fake.

Signal: 2 Next: Run proof scenarios at 100, 500, and 1000+ observations and compare the curves.
aimissing
AI Analyst

Ollama reviews weak evidence, missing measurements, experiment ideas, and next actions.

Signal: not run Next: Use auto-strong for serious reviews; keep deterministic metrics as the source of truth.
planningconnected
Project Scan

Project profiles connect source, docs, tests, firmware, evidence, and planned metrics.

Signal: 1 Next: Run plan-ai after major repo or hardware-documentation changes.
planningconnected
Local Repo Snapshots

Repo snapshots capture Git state, file mix, test/doc/config signals, and local structure for project planning.

Signal: 2 Next: Use Collect Project Snapshot on each project after important work or before a planning review.
planningpartial
Source Connection Plan

Source connectors define which project inputs BETA should expect for issues, CI, docs, bench, field, metrics, and AI sessions.

Signal: 4/7 Next: Register planned sources, then connect real files or folders as they become available.
fieldmissing
Field And Bench Logs

Power, thermal, calibration, endurance, and human acceptance data prove real-world readiness.

Signal: not imported Next: Create a simple CSV or report.json path for bench and field validation evidence.
managementconnected
Project Todo Ledger

Project todos show planned work, active focus, blockers, due dates, and completion evidence.

Signal: 3 Next: Add todos for the next project-management cycle, then mark work doing, blocked, done, or dropped as reality changes.
managementconnected
Work Session Ledger

Structured work sessions explain where time went, what produced evidence, and which blockers or rework consumed effort.

Signal: 4 Next: Record work sessions after meaningful project work with category, minutes, outcome, evidence, blockers, and next step.
managementmissing
Issue And CI History

Project-management evidence needs defects, milestones, build pass rate, coverage, and flaky-test history.

Signal: not imported Next: Add a connector/importer for issues, milestones, CI status, coverage, and release notes.
aiconnected
AI Work Sessions

AI session summaries show what an AI helped change, suggested, tested, or left uncertain.

Signal: 3 Next: Import Codex/ChatGPT/session summaries as source_type=ai-session after meaningful project work.

Measurement Plan

  • P1 Repeatable Ingest Load Curve Proves the system can accept field observations at useful rates without hidden bottlenecks. Metric: observation_throughput_per_s | Current: 0.0 Collect: Run fixed-rate proof scenarios at several observation counts and store every report.json. Done when: Throughput holds or improves across repeated matching runs with no failed required gates.
  • P1 Endpoint Latency Distribution Operator readiness depends on fast reads, not only successful ingest. Metric: query_p95_ms | Current: 0.0 Collect: Record per-endpoint p50/p95/p99 latency during proof and field-like runs. Done when: Query p95 is stable or improving, with endpoint-level evidence explaining outliers.
  • P1 Storage Growth Curve Storage cost must stay bounded as data volume grows beyond tiny demos. Metric: db_bytes_per_observation | Current: 0.0 Collect: Run increasing observation counts and chart bytes per observation over time. Done when: DB bytes per observation stays flat or drops as sample size grows.
  • P2 Field-Link Portal Load Slow links and tunnels need payload and load-time evidence, not only local browser checks. Metric: portal_html_bytes | Current: 0.0 Collect: Measure payload bytes, compressed bytes, first response, and full load time on a constrained link. Done when: Portal payload and load timing remain within the field-readiness threshold.
  • P2 Reliability And Failure History Repeated failures and flaky tests are project-management signals, not just engineering annoyances. Metric: failure_rate Collect: Import CI pass rate, failed proof runs, flaky tests, and repeated-regression counts. Done when: Failure rate trends down and high-priority defects do not repeatedly reappear.
  • P2 Human Acceptance Signoff The proof should connect to whether a real user or reviewer can trust the system. Metric: operator_acceptance Collect: Capture review notes, field-test signoff, defect severity, and acceptance criteria. Done when: Each major claim has at least one human-reviewed acceptance record.
  • P1 Firmware Boot And I/O Trace Firmware evidence is needed for devices that must start reliably and read sensors safely. Metric: firmware_boot_success_rate Collect: Collect serial boot logs, sensor traces, flash/build result, and fault-injection notes. Done when: Firmware boots repeatedly and reports expected I/O without unsafe failures.
  • P1 Firmware Identity And Board Lineage Recent OMEGA firmware emits operator labels, physical board tags, and factory MACs; proof should catch wrong-firmware or wrong-board mistakes. Metric: firmware_identity_fields_present Collect: Capture serial or bridge-ingested frames and verify operator_label, physical_board_tag, physical_mac, node id, and mesh id consistency. Done when: Every tested firmware target emits stable identity fields and the operator can map each frame to the expected physical board.
  • P1 Firmware Power And Thermal Health Recent firmware adds ESP temperature and PMU/battery fields that prove nodes can be monitored during operation. Metric: firmware_health_telemetry_present Collect: Capture boot and heartbeat traces and verify esp_temp_c plus battery_mv, battery_pct, charging/USB power, heap, and radio readiness where supported. Done when: Health telemetry appears in repeated frames without sentinel or missing values on boards that support those sensors.
  • P2 Operator OLED Readability The dense one-screen OLED layout needs human-readable operator evidence, not only code review. Metric: operator_oled_readability Collect: Photograph or record the OLED during boot and heartbeat states; confirm OA label, mesh id, uptime, GPS/health, and counters fit without clipping. Done when: An operator can identify the board and key health state from the OLED in one screen.
  • P1 Required Gate Pass Rate Percent of accepted OMEGA proof reports where required checks pass Metric: required_gate_pass_rate Collect: Record this value in a real report, metrics import, bench log, CI result, or field note. Done when: The metric has a real baseline, a repeat run, and a clear decision rule.
  • P1 Advisory Gate Pass Rate Percent of accepted OMEGA proof reports where advisory performance checks pass Metric: advisory_gate_pass_rate Collect: Record this value in a real report, metrics import, bench log, CI result, or field note. Done when: The metric has a real baseline, a repeat run, and a clear decision rule.
  • P1 Wire Savings Measures binary mesh protocol efficiency versus JSON Metric: wire_savings_percent Collect: Record this value in a real report, metrics import, bench log, CI result, or field note. Done when: The metric has a real baseline, a repeat run, and a clear decision rule.
  • P1 Unit Test Pass Rate Percent of OMEGA pytest tests passing Metric: unit_test_pass_rate | Current: 100.0 % Collect: Run the OMEGA pytest suite via run-tests. Done when: Real baseline, repeat run, clear decision rule.
  • P1 Test Failures Failing OMEGA tests that need fixing Metric: test_failure_count | Current: 0 count Collect: Run the OMEGA pytest suite via run-tests. Done when: Failures trend to zero.
  • P2 Test Pass Rate Baseline correctness and regression signal. Metric: test_pass_rate Collect: Add the metric to future project reports or an external evidence import. Done when: Metric appears in the dashboard with a baseline and trend.

Version Change Since Previous Snapshot

A new data version was generated, but tracked evidence/model/project-config fields did not materially change.

snapshot-only
Current data version beta-2026-06-22t21-24-40z-228d8bfa07 2026-06-22T21:24:40Z
Previous data version beta-2026-06-22t20-02-03z-2fe016a499 2026-06-22T20:02:03Z
  • No material tracked changesThis snapshot did not change the tracked evidence/model/project-config posture.

Data Version History

Each row is a generated dashboard/report snapshot for this scope. Use it to see which BETA version, model, and project config produced past data.

12 shown
Data VersionRecordedBETAAI ModelRealQuarantinedProject Configs
beta-2026-06-22t21-24-4...project:omega 2026-06-22T21:24:40Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t20-02-0...project:omega 2026-06-22T20:02:03Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-52-5...project:omega 2026-06-22T19:52:58Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-51-1...project:omega 2026-06-22T19:51:17Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-51-1...project:omega 2026-06-22T19:51:16Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-41-0...project:omega 2026-06-22T19:41:01Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-41-0...project:omega 2026-06-22T19:41:01Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-37-1...project:omega 2026-06-22T19:37:17Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-37-1...project:omega 2026-06-22T19:37:16Z 0.2.0 none 2 12 omega v3 (explicit)
beta-2026-06-22t19-37-0...project:omega 2026-06-22T19:37:01Z 0.2.0 none 1 12 omega v3 (explicit)
beta-2026-06-22t19-32-0...project:omega 2026-06-22T19:32:09Z 0.2.0 none 1 12 omega v2 (explicit)
beta-2026-06-22t19-32-0...project:omega 2026-06-22T19:32:08Z 0.2.0 none 1 12 omega v2 (explicit)

Run History

RunScenarioGeneratedScoreThroughputQuery p95
omega-automated-test-run-2026-06-22t19-37-16z automated-test-run:0h:0m 2026-06-22T19:37:16Z 61.00 0 obs/s 0 ms
omega-real-underwater-ops-20260606 underwater-ops:6h:60m 2026-06-06T19:05:22Z 93.00 41.19 obs/s 17.38 ms

AI Analyst

AI analysis has not been run for the latest dashboard data.

Data Quality Assessment

Run AI analysis to review data quality.

Actionable Operating Plan

  • P0
    Strengthen claim: The system accepts and stores valid observations correctly. Current claim evidence is thin and affects trust in the system. Owner: QA/project lead | Impact: high | Confidence: medium Success: claim evidence score Evidence: Repeat the matching scenario and add the missing signals listed in claim caveats. Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status.
  • P0
    Strengthen claim: Operators can retrieve useful current state quickly. Current claim evidence is gap and affects trust in the system. Owner: QA/project lead | Impact: high | Confidence: medium Success: claim evidence score Evidence: Repeat the matching scenario and add the missing signals listed in claim caveats. Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status.
  • P1
    Repeatable Ingest Load Curve Proves the system can accept field observations at useful rates without hidden bottlenecks. Owner: Engineering | Impact: high | Confidence: thin Success: Throughput holds or improves across repeated matching runs with no failed required gates. Evidence: A fresh matching proof report plus before/after metric comparison. Run the same scenario used by the comparable baseline. Capture report.json and import it into BETA. Check whether the target metric improved, stabilized, or regressed again. If it regresses again, profile the owning code path before adding new features.
  • P1
    Endpoint Latency Distribution Operator readiness depends on fast reads, not only successful ingest. Owner: Engineering | Impact: high | Confidence: thin Success: Query p95 is stable or improving, with endpoint-level evidence explaining outliers. Evidence: A fresh matching proof report plus before/after metric comparison. Run the same scenario used by the comparable baseline. Capture report.json and import it into BETA. Check whether the target metric improved, stabilized, or regressed again. If it regresses again, profile the owning code path before adding new features.

What To Focus On

  • P1 Repeatable Ingest Load Curve Proves the system can accept field observations at useful rates without hidden bottlenecks. Next: Run fixed-rate proof scenarios at several observation counts and store every report.json. Success: Throughput holds or improves across repeated matching runs with no failed required gates.
  • P1 Endpoint Latency Distribution Operator readiness depends on fast reads, not only successful ingest. Next: Record per-endpoint p50/p95/p99 latency during proof and field-like runs. Success: Query p95 is stable or improving, with endpoint-level evidence explaining outliers.
  • P1 Storage Growth Curve Storage cost must stay bounded as data volume grows beyond tiny demos. Next: Run increasing observation counts and chart bytes per observation over time. Success: DB bytes per observation stays flat or drops as sample size grows.
  • P1 Firmware Boot And I/O Trace Firmware evidence is needed for devices that must start reliably and read sensors safely. Next: Collect serial boot logs, sensor traces, flash/build result, and fault-injection notes. Success: Firmware boots repeatedly and reports expected I/O without unsafe failures.
  • P1 Reduce blocked and rework time The work ledger shows 30.0% of tracked time in blocked, rework, or unclear sessions. Next: Review the top blocker/rework notes, remove one cause, and log the next session outcome. Success: work waste percent below 20%

How To Work Better

  • Use a tight evidence loop This makes progress attributable instead of mixing several changes into one unclear result. Start: Plan one change, run one matching proof scenario, save the report, then review the regression and QA matrix.
  • Keep proof reports small but complete Structured data feeds charts, AI review, reports, and trend analysis automatically. Start: Add metrics as structured report fields instead of notes whenever possible.
  • Separate current-state proof from improvement proof This prevents the tool from overstating progress when it only has a snapshot. Start: Use current values to prove presence, but use matching baselines and repeated runs to prove improvement.
  • Close data-quality gaps before polishing Needed to separate real progress from a single isolated run. Start: Start with: Comparable baseline exists.
  • Review the work ledger every cycle This makes project management based on actual behavior instead of memory or vibes. Start: Look at evidence percent, waste percent, and latest session next steps before picking new work.
  • Review the todo ledger before new work This keeps planning grounded in actual commitments instead of only metric gaps or AI suggestions. Start: Check open, doing, blocked, overdue, and completed-with-evidence todos before picking the next task.

Improvement Opportunities

  • No AI opportunitiesRun AI analysis to generate model-assisted suggestions.

Claim Relevance Review

  • No items yetRun AI analysis to populate this section.

Action Tracker

Actions are built from deterministic BETA guidance, manager risks, measurement gaps, and advisory AI output. Statuses are gated by real evidence availability.

beta.action_tracker.v1
Actions 53

tracked recommendations

Active 46

ready to work

Blocked 0

need real data first

Risks 4

manager risks

Avoid 3

guardrails

AI 0

advisory actions

Showing top 12 of 53 tracked actions for this scope.

StatusActionSourceMetricNext StepEvidence Needed
activeP0 Repeat underwater-ops proof for comparison baselineRun the same underwater-ops proof again after the next OMEGA change so BETA can compare against omega-real-underwater-ops-20260606. Project todo ledgerdeterministic project_todo_progressAt least two accepted underwater-ops real reports appear on the OMEGA progress page. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. artifacts/omega-proof/omega-real-underwater-ops-20260606/report.json
activeP0 Strengthen claim: Operators can retrieve useful current state quickly.Current claim evidence is gap and affects trust in the system. Deterministic operating plandeterministic query_p95_ms, portal_html_bytes, station_featuresclaim evidence score Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status. Repeat the matching scenario and add the missing signals listed in claim caveats.
activeP0 Strengthen claim: The system accepts and stores valid observations correctly.Current claim evidence is thin and affects trust in the system. Deterministic operating plandeterministic write_p95_ms, observation_throughput_per_sclaim evidence score Read the claim caveats and missing signals. Add the missing signal to the next proof run or project evidence import. Rerun the scenario and confirm the claim moves out of thin/gap status. Repeat the matching scenario and add the missing signals listed in claim caveats.
activeP1 Advisory Gate Pass RatePercent of accepted OMEGA proof reports where advisory performance checks pass Measurement plandeterministic advisory_gate_pass_rateThe metric has a real baseline, a repeat run, and a clear decision rule. Record this value in a real report, metrics import, bench log, CI result, or field note. The metric has a real baseline, a repeat run, and a clear decision rule.
activeP1 Advisory Gate Pass RateThis planned metric is not active yet. Manager metric backlogdeterministic advisory_gate_pass_rateThe metric has a real baseline, a repeat run, and a clear decision rule. Add this metric to a real report, CI import, bench log, field note, or manual evidence record. The metric has a real baseline, a repeat run, and a clear decision rule.
activeP1 Capture hardware and field validation evidenceAdd real board, radio, power, thermal, OLED, GNSS, or dockside/field evidence when available. Project todo ledgerdeterministic project_todo_progressAt least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. At least one hardware/field evidence source is connected to OMEGA and referenced by a work log or proof report.
activeP1 Connect AI AnalystOllama reviews weak evidence, missing measurements, experiment ideas, and next actions. Missing data sourcedeterministic aisource connected Use auto-strong for serious reviews; keep deterministic metrics as the source of truth. Use auto-strong for serious reviews; keep deterministic metrics as the source of truth.
activeP1 Connect Comparable BaselineA matching prior scenario separates real progress from one-off numbers. Missing data sourcedeterministic evidencesource connected Repeat the same scenario after changes so every metric has a before/after comparison. Repeat the same scenario after changes so every metric has a before/after comparison.
activeP1 Connect Field And Bench LogsPower, thermal, calibration, endurance, and human acceptance data prove real-world readiness. Missing data sourcedeterministic fieldsource connected Create a simple CSV or report.json path for bench and field validation evidence. Create a simple CSV or report.json path for bench and field validation evidence.
activeP1 Connect Issue And CI HistoryProject-management evidence needs defects, milestones, build pass rate, coverage, and flaky-test history. Missing data sourcedeterministic managementsource connected Add a connector/importer for issues, milestones, CI status, coverage, and release notes. Add a connector/importer for issues, milestones, CI status, coverage, and release notes.
activeP1 Connect OMEGA CI and test historyAttach current OMEGA test/proof workflow outputs, CI runs, and relevant pytest results as BETA sources. Project todo ledgerdeterministic project_todo_progressOMEGA source coverage shows CI/test history as connected and manager risks stop asking for it. Mark it doing, done with evidence, blocked with a blocker, or dropped with a reason. OMEGA source coverage shows CI/test history as connected and manager risks stop asking for it.
activeP1 Connect Quarantined Demo ReportsSynthetic, demo, smoke, fixture, and local proof reports stay visible for audit but are excluded from metrics. Missing data sourcedeterministic evidencesource connected Replace quarantined reports with real evidence or mark real reports explicitly with metadata.data_authenticity=real. Replace quarantined reports with real evidence or mark real reports explicitly with metadata.data_authenticity=real.

AI can suggest actions, but BETA only treats metrics, imported inputs, proof reports, and work logs as evidence.