Job Description
What does a great Data Architect do?
This job will require the candidate to design and build cloud infrastructure, data processing and analytics environments to be used by various groups developing, executing and supporting data and analytics functions. Deep knowledge of cloud, computing for analytics and database technologies, good communication and organization and communication skills. Platform used in system development, data management & collaboration, growth potential to either other data platform or data management roles.
What you will do:
- Create and maintain documentation such as Standard Operating Procedures on -call, cloud elevations,high level platform scalability work flow diagrams, process and procedural documents for implementation and support teams to support.
- Use AWS Cloud Formation templates to create custom sized EC2 instances, VPC, subnets, NAT, ELB and Security groups.
- Setup IAM for installing configuring and troubleshooting on various Amazon images for server migration from physical into cloud.
- Work on AWS cloud services like EC2, s3, RDS, ELBS, VPC, Route53, Auto Scaling groups(asg), CloudWatch, CloudFront.
- Created alarms and trigger points in CloudWatch based on thresholds and monitored the server's performance, CPU Utilization, disk usage and Utilized AWS CloudWatch services to monitor environment for operational & performance metrics during load testing.
- Experience in implementing Data warehouse solutions in AWS Redshift. Work on various projects to migrate data from one database to AWS Redshift, RDS, ELB, EMR, Dynamo DB and S3.
- Troubleshoot high-load, memory, network, CPU usage with Nagios, cacti, ELK, New Relic.
- Creating Jenkins pipeline jobs for Puppet release process for module deployment, using Kanban AGILE methodology for puppet development.
- Install and configure of an automated tool Puppet that included installation and configuration of Puppet Master, Agent nodes and an admin control workstation.
- Manage AWS EC2 instances utilizing Auto Scaling, Elastic Load Balancing and Glacier for our QA and UAT environments as well as infrastructure servers for GIT/Bitbucket and Puppet
- Implement continuous integration web hooks and workflows around Jenkins to automate the dev test deploy workflow around Puppet codebase.
- Integrate Splunk with AWS deployment using puppet to collect data from all EC2 systems into Splunk.
- Work on Terraform 9 and 11 to automate the AWS services, implemented automation for SDLC governance. And develop Python scripts to automate deployment process using Jenkins, launch pad apps for ECS using Docker containers docker, terraform, Splunk, puppet, Jenkins, AWS, Azure, GCP, antifactory, ansible, Kubernetes, snowflake, harness.
What you will need to have:
- Bachelors in computer science.
- OS: Linux, Shell.
- DevOps Infrastructure: Terraform, Ansible, Antifactory, docker, terraform, Splunk, puppet, Jenkins, AWS, Azure, GCP, antifactory, ansible, Kubernetes, GitHub, jira, harness,
- Cloud: AWS, Azure, GCP (plus).
- Infrastructure: Kubernetes Containers Understand scaling, ha, dr concepts.
- Coding: Terraform, Python, Understanding/implementation on api, Java (nice to have), SQL.
- Networking, general app admin, user/group admin.
What would be great to have:
- Experience: Any data, parallel processing, database experience, data science, data engineering (all pluses).
- Configuration management (nice to have): Ansible Chef Puppet.
- Experience with databases: e.g. SQL server, Oracle, Snowflake, MySQL.
- Internet data sources, Jason, XML, csv, etc.