Job Description
Analyze and organize raw data from multiple sources.
Design and build data systems and pipelines using tools like Apache NiFi.
Evaluate business needs and translate them into technical requirements.
Interpret trends and patterns through complex data analysis and reporting.
Prepare data for prescriptive and predictive modeling. Develop algorithms and prototypes for data-driven solutions.
Enhance data quality and reliability using Spark and schedule PySpark jobs with Airflow.
Identify opportunities for data acquisition and integration. Build analytical tools and programs to support business objectives.
Collaborate with data scientists and architects on various projects.
Key Responsibilities
Experience: 4–6 years as a Data Engineer.
Strong technical expertise in data models, data mining, and segmentation techniques.
Proficiency in programming languages such as Java,Python and Spark
Hands-on experience with SQL database design and optimization.
Solid numerical and analytical skills.
apache kafka & apache spark streaming is a plus.
Data engineering certification (e.g., IBM Certified Data Engineer) is a plus.
Preferred Tools & Technologies: Apache NiFi, Spark, Airflow SQL, MySQL Java, Python Cloud platforms (AWS, GCP, or Azure)
Qualifications
Bachelor’s degree in Computer Science, IT, or related field (Master’s preferred).