ETL Big Data Developer

in Rockville, MD

Job #:277298

Practice Area: IT

Date Posted: 10/01/2019

Share this job  

Job Title:  Big Data Developer- ETL

Location:  Rockville, MD

Industry:  Financial/ IT

Terms:  Contract / Contract-to-Hire

Job Summary:

  • The Data Platform is responsible for the collection, management and dissemination of data. It’s an enterprise capability that services multiple business lines across the entire organization. The platform collects and processes billions of records (up to 100 billion) each day in support of regulatory mission to protect investors and promote market integrity. We are embarking upon a large, multi-year, enterprise-wide initiative to re-platform major pieces of our existing offering, replacing legacy functionality with a set of modern, highly customer-centric products and capabilities.
    Job Responsibilities:
  • Understand complex business requirements
  • Design and develop ETL pipeline for collecting, validating and transforming data according to the specification
  • Develop unit tests and performance tests
  • Maintain optimal data pipeline architecture
  • Design ETL jobs for optimal execution in AWS cloud environment
  • Reduce processing time and cost of ETL workloads
  • Participate in peer reviews and design/code review meetings
  • Provide support for production support operations team
  • Implement data quality checks.
  • Identify areas where machine learning can be used to identify data anomalies

Job Requirements:

  • 10+ years of experience in programming language Java or Scala
  • 10+ years of experience in ETL projects
  • 3+ years of experience in big data projects
  • Believes in Scrum/Agile, and has deep experience delivering software when working on teams that use Scrum/Agile methodology
  • Strong and creative analytical and problem-solving skills
  • Strong experience in Java or Scala
  • Strong experience in big data technologies like AWS EMR, Hadoop, Hive, Apache Spark, Apache Hbase, AWS Dynamo DB, AWS Lambda
  • Must be able to write complex SQL queries
  • Experience in performance tuning and optimization
  • Familiar with columnar storage formats ORC, Parquet and various compression techniques
  • Experience in writing Unix shell scripts
  • Unit testing using Junit or Scalatest
  • Git/Maven/Gradle
  • Code Reviews
  • Agile

    The following skills are a plus:

  • AWS Cloud
  • BPM/Activiti
  • Python scripting

 

Frequently Asked Questions:

  • Benefits?  Yes, provided by ettain group
  • Remote Workers?  Not eligible, must work onsite with the team
  • Visa Sponsorship?  Yes, sponsorship is being considered
  • Corp-to-Corp?  Yes, C2C candidates are being considered

 

How to Apply:

For immediate attention please apply online

 

ettain group is a talent solutions company dedicated to matching people with the most relevant employment opportunities in IT, healthcare IT and the digital creative space. Top employers and highly skilled talent throughout the U.S. select ettain group for a better recruitment experience. Our culture empowers teams with the flexibility to deliver a superior candidate experience and client experience using market intelligence. We recruit resources, own projects and manage programs to offer a wide range of IT development and talent solutions to our clients and candidates. To learn more about ettain group visit www.ettaingroup.com. To explore more job opportunities with ettain group, visit www.ettaingroup.com/job-board.aspx
Apply Now