Extend the platform,
empower your team.
Get insights into Google Cloud Composer metrics collected from the Google Operations API to ensure health of your cloud infrastructure.
ExtensionThis Dynatrace extension leverages data collected from the Google Operations API to constantly monitor health and performance of Google Cloud Composer services. This extension combines all relevant data into pre-configured dashboards and provides alerting and event tracking.
View and analyze 17 metrics that are specific to Google Cloud Composer, like API Requests, API Latency, Parse Error Count, and more.
Build custom dashboards for your cloud infrastructure.
Analyze Google Cloud Composer logs.
Set custom alerts that trigger remediation workflows.
Google Cloud Composer metric and log ingestion requires advanced GCP integration.
This extension package contains:
To provide correlation and causation analysis all ingested metrics and logs are analyzed by the Dynatrace Davis AI engine, which consumes DDUs.
Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual feature sets can be activated and deactivated by your administrator during configuration.
Metric name | Metric key | Description | Unit |
---|---|---|---|
Active Scheduler instances | cloud.gcp.composer_googleapis_com.environment.active_schedulers | - | Count |
Active Triggerers instances | cloud.gcp.composer_googleapis_com.environment.active_triggerers | - | Count |
Active Webserver instances | cloud.gcp.composer_googleapis_com.environment.active_webservers | - | Count |
API Requests | cloud.gcp.composer_googleapis_com.environment.api.request_count | - | Count |
API Latency | cloud.gcp.composer_googleapis_com.environment.api.request_latencies | - | MilliSecond |
Celery task non-zero exit codes | cloud.gcp.composer_googleapis_com.environment.celery.execute_command_failure_count | - | Count |
Celery task publish timeouts | cloud.gcp.composer_googleapis_com.environment.celery.task_timeout_error_count | - | Count |
Serialized DAG fetch duration | cloud.gcp.composer_googleapis_com.environment.collect_db_dag_duration | - | MilliSecond |
DAG callback exceptions | cloud.gcp.composer_googleapis_com.environment.dag_callback.exception_count | - | Count |
DAG refresh errors | cloud.gcp.composer_googleapis_com.environment.dag_file.refresh_error_count | - | Count |
DAG file load time | cloud.gcp.composer_googleapis_com.environment.dag_processing.last_duration | - | MilliSecond |
Time since DAG processing | cloud.gcp.composer_googleapis_com.environment.dag_processing.last_run_elapsed_time | - | Second |
DagFileProcessorManager stall count | cloud.gcp.composer_googleapis_com.environment.dag_processing.manager_stall_count | - | Count |
Parse Error Count | cloud.gcp.composer_googleapis_com.environment.dag_processing.parse_error_count | - | Count |
DAG parsing processes | cloud.gcp.composer_googleapis_com.environment.dag_processing.processes | - | Count |
Processors Timeout Count | cloud.gcp.composer_googleapis_com.environment.dag_processing.processor_timeout_count | - | Count |
Total Parse Time | cloud.gcp.composer_googleapis_com.environment.dag_processing.total_parse_time | - | Second |
Dag Bag Size | cloud.gcp.composer_googleapis_com.environment.dagbag_size | - | Count |
Airflow metadata database size | cloud.gcp.composer_googleapis_com.environment.database.airflow.size | - | Byte |
Auto-failover request count | cloud.gcp.composer_googleapis_com.environment.database.auto_failover_request_count | - | Count |
Available for failover | cloud.gcp.composer_googleapis_com.environment.database.available_for_failover | - | Count |
Database CPU reserved cores | cloud.gcp.composer_googleapis_com.environment.database.cpu.reserved_cores | - | Count |
Database CPU usage time | cloud.gcp.composer_googleapis_com.environment.database.cpu.usage_time | - | Second |
Database CPU utilization | cloud.gcp.composer_googleapis_com.environment.database.cpu.utilization | - | Count |
Database disk usage | cloud.gcp.composer_googleapis_com.environment.database.disk.bytes_used | - | Byte |
Database disk quota | cloud.gcp.composer_googleapis_com.environment.database.disk.quota | - | Byte |
Database disk utilization | cloud.gcp.composer_googleapis_com.environment.database.disk.utilization | - | Count |
Database memory usage | cloud.gcp.composer_googleapis_com.environment.database.memory.bytes_used | - | Byte |
Database memory quota | cloud.gcp.composer_googleapis_com.environment.database.memory.quota | - | Byte |
Database memory utilization | cloud.gcp.composer_googleapis_com.environment.database.memory.utilization | - | Count |
Database connections | cloud.gcp.composer_googleapis_com.environment.database.network.connections | - | Count |
Database connections limit | cloud.gcp.composer_googleapis_com.environment.database.network.max_connections | - | Count |
Database received bytes | cloud.gcp.composer_googleapis_com.environment.database.network.received_bytes_count | - | Byte |
Database sent bytes | cloud.gcp.composer_googleapis_com.environment.database.network.sent_bytes_count | - | Byte |
Database Healthy | cloud.gcp.composer_googleapis_com.environment.database_health | - | Unspecified |
Failed SLA miss email notifications | cloud.gcp.composer_googleapis_com.environment.email.sla_notification_failure_count | - | Count |
Executor Open Slots | cloud.gcp.composer_googleapis_com.environment.executor.open_slots | - | Count |
Executor Queued Tasks | cloud.gcp.composer_googleapis_com.environment.executor.queued_tasks | - | Count |
Executor Running Tasks | cloud.gcp.composer_googleapis_com.environment.executor.running_tasks | - | Count |
Finished Task Instance Count | cloud.gcp.composer_googleapis_com.environment.finished_task_instance_count | - | Count |
Airflow api checks count | cloud.gcp.composer_googleapis_com.environment.health.airflow_api_check_count | - | Count |
Autoscaling components checks count | cloud.gcp.composer_googleapis_com.environment.health.autoscaling_check_count | - | Count |
CMEK encryption checks count | cloud.gcp.composer_googleapis_com.environment.health.cmek_encryption_check_count | - | Count |
Container restarts count. | cloud.gcp.composer_googleapis_com.environment.health.container_restart_count | - | Count |
Dependency checks count | cloud.gcp.composer_googleapis_com.environment.health.dependency_check_count | - | Count |
Dependency permissions checks count | cloud.gcp.composer_googleapis_com.environment.health.dependency_permissions_check_count | - | Count |
Pod event count | cloud.gcp.composer_googleapis_com.environment.health.pod_event_count | - | Count |
Redis queue checks count | cloud.gcp.composer_googleapis_com.environment.health.redis_queue_check_count | - | Count |
Healthy | cloud.gcp.composer_googleapis_com.environment.healthy | - | Unspecified |
Job count | cloud.gcp.composer_googleapis_com.environment.job.count | - | Count |
Failed Job heartbeats | cloud.gcp.composer_googleapis_com.environment.job.heartbeat_failure_count | - | Count |
Maintenance operation | cloud.gcp.composer_googleapis_com.environment.maintenance_operation | - | Count |
Celery Workers | cloud.gcp.composer_googleapis_com.environment.num_celery_workers | - | Count |
Tasks created per operator | cloud.gcp.composer_googleapis_com.environment.operator.created_task_instance_count | - | Count |
Operator executions | cloud.gcp.composer_googleapis_com.environment.operator.finished_task_instance_count | - | Count |
Open slots in the pool | cloud.gcp.composer_googleapis_com.environment.pool.open_slots | - | Count |
Queued slots in the pool | cloud.gcp.composer_googleapis_com.environment.pool.queued_slots | - | Count |
Running slots in the pool | cloud.gcp.composer_googleapis_com.environment.pool.running_slots | - | Count |
Starving tasks in the pool | cloud.gcp.composer_googleapis_com.environment.pool.starving_tasks | - | Count |
Critical section duration | cloud.gcp.composer_googleapis_com.environment.scheduler.critical_section_duration | - | MilliSecond |
Critical section lock failures | cloud.gcp.composer_googleapis_com.environment.scheduler.critical_section_lock_failure_count | - | Count |
Scheduler Pod Eviction Count | cloud.gcp.composer_googleapis_com.environment.scheduler.pod_eviction_count | - | Count |
Externally killed tasks | cloud.gcp.composer_googleapis_com.environment.scheduler.task.externally_killed_count | - | Count |
Orphaned task count | cloud.gcp.composer_googleapis_com.environment.scheduler.task.orphan_count | - | Count |
Scheduler managed tasks | cloud.gcp.composer_googleapis_com.environment.scheduler.tasks | - | Count |
Scheduler Heartbeats | cloud.gcp.composer_googleapis_com.environment.scheduler_heartbeat_count | - | Count |
Failed SLA callback notifications | cloud.gcp.composer_googleapis_com.environment.sla_callback_notification_failure_count | - | Count |
Smart sensor poking exception failures | cloud.gcp.composer_googleapis_com.environment.smart_sensor.exception_failures | - | Count |
Smart sensor poking infrastructure failures | cloud.gcp.composer_googleapis_com.environment.smart_sensor.infra_failures | - | Count |
Smart sensor poking exceptions | cloud.gcp.composer_googleapis_com.environment.smart_sensor.poked_exception | - | Count |
Smart sensor successfully poked tasks | cloud.gcp.composer_googleapis_com.environment.smart_sensor.poked_success | - | Count |
Smart sensor poked tasks | cloud.gcp.composer_googleapis_com.environment.smart_sensor.poked_tasks | - | Count |
Snapshot creation count | cloud.gcp.composer_googleapis_com.environment.snapshot.creation_count | - | Count |
Snapshot creation elapsed time | cloud.gcp.composer_googleapis_com.environment.snapshot.creation_elapsed_time | - | Second |
Snapshot size | cloud.gcp.composer_googleapis_com.environment.snapshot.size | - | Byte |
Previously succeeded task instances | cloud.gcp.composer_googleapis_com.environment.task_instance.previously_succeeded_count | - | Count |
Task Queue Length | cloud.gcp.composer_googleapis_com.environment.task_queue_length | - | Count |
Failed triggers | cloud.gcp.composer_googleapis_com.environment.trigger.failed_count | - | Count |
Succeeded triggers | cloud.gcp.composer_googleapis_com.environment.trigger.succeeded_count | - | Count |
Unfinished Task Instances | cloud.gcp.composer_googleapis_com.environment.unfinished_task_instances | - | Count |
Web server CPU reserved cores | cloud.gcp.composer_googleapis_com.environment.web_server.cpu.reserved_cores | - | Count |
Web server CPU usage time | cloud.gcp.composer_googleapis_com.environment.web_server.cpu.usage_time | - | Second |
Web server health | cloud.gcp.composer_googleapis_com.environment.web_server.health | - | Unspecified |
Web server memory usage | cloud.gcp.composer_googleapis_com.environment.web_server.memory.bytes_used | - | Byte |
Web server memory quota | cloud.gcp.composer_googleapis_com.environment.web_server.memory.quota | - | Byte |
Maximum Airflow workers | cloud.gcp.composer_googleapis_com.environment.worker.max_workers | - | Count |
Minimum Airflow workers | cloud.gcp.composer_googleapis_com.environment.worker.min_workers | - | Count |
Worker Pod Eviction Count | cloud.gcp.composer_googleapis_com.environment.worker.pod_eviction_count | - | Count |
Workers scale factor target | cloud.gcp.composer_googleapis_com.environment.worker.scale_factor_target | - | Count |
Zombie Tasks Killed | cloud.gcp.composer_googleapis_com.environment.zombie_task_killed_count | - | Count |
Support for GCP overview
No release notes