Probable root causes
Ranked by how much of the current breakage each one explains.
- Workload innovation/reqiq has no ready replicas1 component downstream
This workload is serving no traffic. Check its pods with 'kubectl get pods -n innovation -l' matching its selector, then describe the failing pod.
- Broken route — nothing is listening on port 30501 component downstream
Start the service that is supposed to own port 3050, or remove the stale nginx route if the application has been retired. Check 'docker ps -a' and 'systemctl list-units --state=failed' on this host — a container or unit that exited is the usual cause.
- Broken route — nothing is listening on port 30511 component downstream
Start the service that is supposed to own port 3051, or remove the stale nginx route if the application has been retired. Check 'docker ps -a' and 'systemctl list-units --state=failed' on this host — a container or unit that exited is the usual cause.
- Broken route — nothing is listening on port 30601 component downstream
Start the service that is supposed to own port 3060, or remove the stale nginx route if the application has been retired. Check 'docker ps -a' and 'systemctl list-units --state=failed' on this host — a container or unit that exited is the usual cause.
Production — KVM8
docker · kubernetes · systemd · nginx
- Workloads
- 33
- CPU
- 8 vCPU
- Memory free
- 26.5 of 31.3 GB
- Disk free
- 306.1 GB
- Kubernetes
- v1.28.15
- Datastores
- 9 reachable, 2 blocked
- homechore.ogmworld.com
- innovation.ogmworld.com
- inspector.ogmworld.com
- investor.ogmworld.com
- ip10.ogmworld.com
- meet.ogmworld.com
- +9 more
Stage — KVM2
kubernetes · systemd · nginx
- Workloads
- 13
- CPU
- 2 vCPU
- Memory free
- 5.8 of 7.8 GB
- Disk free
- 73.7 GB
- Kubernetes
- v1.28.15
- Datastores
- 9 reachable, 2 blocked
- stage.inspector.ogmworld.com
- stage.ogmworldwide.com
- stage.problem.guide
- stage.samskriti.ogmworld.com
Development — KVM4
docker · systemd · nginx
- Workloads
- 7
- CPU
- 4 vCPU
- Memory free
- 13.7 of 15.6 GB
- Disk free
- 154.8 GB
- Datastores
- 8 reachable, 3 blocked
- dev.cd.ogmworld.com
- dev.inspector.ogmworld.com
- dev.ogmworldwide.com
- dev.problem.guide
- dev.samskriti.ogmworld.com
Datacenter — shared data
docker · systemd
- Workloads
- 31
- CPU
- 8 vCPU
- Memory free
- 25.7 of 31.3 GB
- Disk free
- 304.6 GB
Broken public routes
nginx forwards these hostnames to a port with no process listening — visitors get a 502.
| Environment | Backend | Reason |
|---|---|---|
| prod | 127.0.0.1:3050 | nginx forwards ip10.ogmworld.com/ here, but nothing is listening on port 3050 on this host. |
| prod | 127.0.0.1:3051 | nginx forwards investor.ogmworld.com/ here, but nothing is listening on port 3051 on this host. |
| prod | 127.0.0.1:3060 | nginx forwards homechore.ogmworld.com/_next/static/ here, but nothing is listening on port 3060 on this host. |
| prod | 127.0.0.1:3070 | nginx forwards sentinelloop.ogmworld.com/_next/static/ here, but nothing is listening on port 3070 on this host. |
| prod | 127.0.0.1:8090 | nginx forwards ip10.ogmworld.com^~ /api/ here, but nothing is listening on port 8090 on this host. |
| prod | 127.0.0.1:8091 | nginx forwards innovation.ogmworld.com/reqiq/ here, but nothing is listening on port 8091 on this host. |
| prod | 127.0.0.1:8092 | nginx forwards investor.ogmworld.com^~ /api/ here, but nothing is listening on port 8092 on this host. |