Infra Notes

Infra Notes

Small operational notes for issues that are easy to forget and expensive to debug twice.

Compliance
Compliance evidence is easier when logging and backup are designed early
Audit evidence becomes easier when logging, monitoring, backup, restore testing, and access control are part of the infrastructure design from the beginning.
ComplianceISO 27001PCI DSSLoggingBackupAudit Evidence
Operations
Backup exists, but restore was never tested
A backup is only useful if the restore path has been tested, documented, and validated against the expected recovery scenario.
BackupRestoreReliabilityDisaster RecoveryCompliance
Cloud Security
Access review is painful when IAM naming is not standardized
Access reviews become harder when IAM users, roles, groups, policies, and permission sets are named without a clear convention.
IAMAccess ReviewCloud SecurityComplianceStandardization
GCP
Service account key creation disabled: check organization policy first
When service account key creation is blocked, the reason may be an organization policy even if project-level IAM looks correct.
GCPIAMOrganization PolicyService AccountCloud Security
Wazuh
Wazuh FIM reached 100000 files: don't raise the limit blindly
Exclude noisy paths first before increasing file_limit on busy Linux servers or container hosts.
WazuhFIMLinuxTuningContainerMonitoring
Wazuh
Wazuh custom rule not triggering: test with real log samples first
Before changing rule logic repeatedly, test custom rules with real log samples and confirm the matched decoder or base rule.
WazuhRulesDecoderTestingSIEM
Kubernetes
PodMonitor exists, but Prometheus still does not scrape the target
A quick checklist for debugging PodMonitor issues when the object exists but the target does not appear in Prometheus.
KubernetesPrometheusPodMonitorObservabilityDebugging
GitOps
ArgoCD rollback should start from Git, not kubectl
In GitOps-managed environments, rollback should start from Git history instead of manually patching Kubernetes resources.
ArgoCDGitOpsRollbackKubernetes
Cloudflare
Cloudflare proxied DNS vs DNS-only: when origin testing gets confusing
Cloudflare proxied records can hide origin behavior, so origin testing should be done explicitly.
CloudflareDNSProxyOriginDebugging
Cloudflare
Cloudflare WAF allowlist should be scoped to hostname, not everything
A WAF allowlist is safer when scoped to specific hostnames instead of the entire zone.
CloudflareWAFAccess ControlDNSCloud Security
Automation
Ansible first_found makes environment-specific config cleaner
Use Ansible first_found to keep host-specific, environment-specific, and default config fallback clean.
AnsibleConfig ManagementRolesAutomationMaintainability
Automation
Wrapper roles let you use upstream Ansible roles without forking everything
Wrapper roles help keep upstream roles clean while adding your own deployment logic, files, and operational flow.
AnsibleRolesMaintainabilityAutomationUpstream
Malware Lab
ClamAV EICAR test not triggering because clamd cache hides it
If repeated EICAR tests stop triggering, clamd cache may be hiding repeated detections.
ClamAVEICARMalware TestCacheWazuh
Observability
Grafana shows no data, but Prometheus already has the metric
Before changing exporters or scrape configs, confirm whether the issue is actually the Grafana query, variable, or label filter.
GrafanaPrometheusPromQLDashboardDebugging
Engineering Practice
Good infra work is boring, repeatable, and easy to hand over
The best infrastructure work is often not flashy. It is predictable, documented, repeatable, and easy for the next person to operate.
InfrastructureMaintainabilityOperationsDocumentationDevOps
AWS
AWS configure sso profile and login
A quick note for setting up an AWS SSO profile and logging in explicitly with the intended profile name.
SSOAWS CLIProfile