Senior Data Engineer Job in Prescience Decision Solutions

Job Description Collaborate with business stakeholders and other technical team members to acquire and migrate data sources that are most relevant to business needs and goals. Demonstrate deep technical and domain knowledge of relational and non-relational databases, Data Warehouses, Data lakes, among other structured and unstructured storage options. Determine solutions that are best suited to develop a pipeline for a particular data source. Develop data flow pipelines to extract, transform, and load data from various data sources in various forms, including custom ETL pipelines that enable model and product development. Write custom scripts to extract data from unstructured / semi-structured sources. Provide clear documentation for delivered solutions and processes, integrating documentation with the appropriate corporate stakeholders. Identify and implement internal process improvements for data management (automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability). Stay current with and adopt new tools and applications to ensure high quality and efficient solutions. Build a cross-platform data strategy to aggregate multiple sources and process development datasets. Requirements Professional Experience of ETL design, development, and performance tuning using ETL tools such as ADF in a multi-dimensional Data Warehousing environment. Hands-on experience in advanced SQL Programming: PL/SQL, T-SQL Should have strong and extensive hands-on experience in Azure, preferably data heavy / analytics applications leveraging relational and NoSQL databases, Data Warehouse and Big Data Expertise with Azure Data Factory, Azure Synapse Analytics, Azure Analysis Services, Azure Data bricks, Blob Storage, Data bricks/Spark, Azure SQL DW/Synapse, and Azure functions. Working experience with Python, Spark, Pyspark. Experience in defining and enabling data quality standards for auditing, and monitoring. Strong analytical abilities and a strong intellectual curiosity In-depth knowledge of relational database design, data warehousing and dimensional data modeling concepts. Strong collaboration and teamwork skills & excellent written and verbal communication skills. Self-starter and motivated with the ability to work in a fast-paced development environment. Proficiency in the development environment, including IDE, database server, GIT, Continuous Integration, unit-testing tool, and defect management tools. Benefits Competitive salary and performance-based bonuses. Comprehensive insurance plans. Collaborative and supportive work environment. Chance to learn and grow with a talented team A positive and fun work environment.