Senior Data Engineer - Python Job at PLANIT Group, Washington DC

K2k1NUFtcURtVm5kaC9TK2Q2Z014bFdKU3c9PQ==
  • PLANIT Group
  • Washington DC

Job Description

Washington, DC Contract On-Site Flexibility/Remote: Seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. Your expertise in Python, PySpark, ETL processes, CI/CD (Jenkins or GitHub), and experience with both streaming and batch workflows will be essential in ensuring the efficient flow and processing of data to support our clients. Responsibilities: Data Architecture and Design: Collaborate with cross-functional teams to understand data requirements and design robust data architecture solutions. Develop data models and schema designs to optimize data storage and retrieval. ETL Development: Implement ETL processes to extract, transform, and load data from various sources. Ensure data quality, integrity, and consistency throughout the ETL pipeline. Python and PySpark Development: Utilize your expertise in Python and PySpark to develop efficient data processing and analysis scripts. Optimize code for performance and scalability, keeping up-to-date with the latest industry best practices. Data Integration: Integrate data from different systems and sources to provide a unified view for analytical purposes. Collaborate with data scientists and analysts to implement solutions that meet their data integration needs. Streaming and Batch Workflows: Design and implement streaming workflows using PySpark Streaming or other relevant technologies. Develop batch processing workflows for large-scale data processing and analysis. CI/CD Implementation: Implement and maintain continuous integration and continuous deployment (CI/CD) pipelines using Jenkins or GitHub Actions. Automate testing, code deployment, and monitoring processes to ensure the reliability of data pipelines. Qualifications: Bachelor's or master's degree in computer science, Information Technology, or a related field. 7+ years of proven experience as a Data Engineer or similar role. Strong programming skills in Python and expertise in PySpark for both batch and streaming data processing. Hands-on experience with ETL tools and processes. Familiarity with CI/CD tools such as Jenkins or GitHub Actions. Solid understanding of data modeling, database design, and data warehousing concepts. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. Preferred Skills: Knowledge of cloud platforms such as AWS, Azure, or Google Cloud. Experience with version control systems (e.g., Git). #J-18808-Ljbffr PLANIT Group

Job Tags

Contract work,

Similar Jobs

MasterCorp

Housekeeping Houseperson (full/part time) Job at MasterCorp

Join Our Team at MasterCorp, Inc.! At MasterCorp, Inc., we provide exceptional service and innovative solutions in the hospitality industry. As a leader in our field, we believe in the power of teamwork, integrity, and a commitment to excellence. Our dynamic and inclusive...

Century Sign Builders

Project Manager (Experienced, On Site) Job at Century Sign Builders

 ...solving? If so, you could be our next Project Manager. Does this describe you?...  ...~ You can visualize the end result and work backwards to achieve that goal by detailed...  ...office role with flexibility to work from home as needed Relocation assistance available... 

T.J. Maxx

Part Time Retail Associate Job at T.J. Maxx

 ...T.J. Maxx - 3200 N. Federal Highway, HomeGoods 149, Fort Lauderdale, FL 33306, United States of America [Retail Sales / Team Member] As a Retail Associate at T.J. Maxx, you'll: Role model established customer experience practices; Support and embody a positive store culture... 

The Guitar Center Company

Merchandise Assistant (Keys, DJ, Lighting) Job at The Guitar Center Company

 ...Key Responsibilities: We are seeking entrepreneurial and analytical candidates for a Merchandising Assistant. You will join our dynamic merchandising team. The ideal candidate will play a crucial role in supporting product creation, managing digital assets, and ensuring... 

Marriott International, Inc

Plant Operator Job at Marriott International, Inc

 ...Located Remotely? N Position Type Non-Management POSITION SUMMARY Program and monitor energy management equipment to operate at maximum operational efficiency and avoid peak demand charges and penalties. Lead and train the heating, ventilation and air...