Dutech’s Job

We are seeking an experienced Senior Databricks Administrator to manage and optimize our Databricks platform in an AWS cloud environment. The ideal candidate will have strong expertise in Databricks workspace administration, Apache Spark, cloud integrations, and data platform governance.

This role focuses on ensuring platform performance, security, scalability, and cost optimization for enterprise data and AI workloads.

Key Responsibilities:

• Administer and manage Databricks workspaces in AWS cloud environments
• Configure and optimize Databricks clusters, job scheduling, and workspace settings
• Manage user access, roles, and permissions using IAM, SCIM, and RBAC
• Implement and enforce cluster policies and governance standards
• Monitor platform health, performance, and availability
• Optimize Apache Spark workloads for performance and scalability
• Integrate Databricks with cloud storage services (e.g., Amazon S3)
• Manage and support Databricks SQL, notebooks, and job orchestration
• Ensure data security, encryption, and compliance requirements are met
• Automate infrastructure and workflows using Terraform, scripting, and CI/CD pipelines
• Support AI/ML workloads using Databricks ML and MLflow (as needed)

Required Qualifications:

• 8+ years of experience in Databricks administration in AWS environments
• Strong expertise in Databricks cluster configuration and workspace management
• Hands-on experience with Apache Spark (performance tuning & troubleshooting)
• Experience managing IAM, SCIM, and role-based access control (RBAC)
• Experience integrating Databricks with cloud storage (S3 or similar)
• Strong understanding of data security, encryption, and compliance
• Experience with DevOps tools (Terraform, CI/CD pipelines, scripting)

Preferred Qualifications:

• Experience in enterprise or government environments
• Familiarity with Databricks Unity Catalog for governance
• Experience with cost optimization strategies for Databricks workloads
• Knowledge of data lake/lakehouse architectures
• Experience supporting AI/ML workloads (MLflow)
• Programming knowledge in Python, SQL, or Scala