Job Description:

Technologies: Azure · NLP · RAG · Semantic Matching · Python

Role Summary

We are looking for a Data Scientist with expertise in Python, Azure Cloud, and NLP to build and enhance machine learning models at scale. The role includes embedding optimisation, semantic matching, LDA and RAG architectures, dense and sparse retrieval pipelines, and migration of cloud-native data pipelines to Azure Databricks.

Core Requirements

• Design and execute end-to-end machine learning pipelines including data extraction, preprocessing, feature engineering, model development, tuning, and deployment.
• Develop machine learning pipelines using Azure Synapse, Databricks, and Snowflake.
• Build and deploy classification, regression, and clustering models.
• Develop and deploy proof-of-concept solutions for client use cases.
• Implement semantic matching and similarity search using cosine similarity, dot-product scoring, and bi-encoder/cross-encoder architectures (e.g., SBERT, sentence-transformers).
• Build embedding models by fine-tuning pre-trained models and optimising embedding storage in vector databases such as Chroma DB, FAISS, and Azure AI Search.

Model Development & Optimisation

• Train and optimise models for new data providers with dynamic input handling.
• Improve LDA model performance for large-scale topic modelling.
• Implement hybrid semantic search by combining dense and sparse retrieval methods.
• Optimise RAG architectures and retrieval QA systems for chatbot and recommendation performance.
• Enable semantic query understanding using intent classification and query expansion techniques.

Forecasting & NLP

• Develop forecasting models for marketing, demand prediction, and trend analysis.
• Apply NLP-based forecasting techniques using sentiment and external data.
• Use semantic similarity for audience intelligence, including zero-shot and few-shot classification techniques.

Data Pipeline & Cloud Migration

• Migrate data pipelines from Azure Synapse to Azure Databricks and retrain models accordingly.
• Optimise embedding storage and retrieval within Azure AI Search.
• Perform vector index tuning including HNSW optimisation and ANN benchmarking for production systems.

Required Skills & Tools

Python, Azure Databricks, Azure ML, Azure Synapse, Azure Blob Storage, Scikit-learn, NumPy, Pandas, Hugging Face, sentence-transformers, FAISS, Chroma DB, Azure AI Search, LangChain, TensorFlow, PyTorch, Statsmodels, Azure OpenAI.

Location:

DGS India - Mumbai - Thane Ashar IT Park

Brand:

Merkle

Time Type:

Full time

Contract Type:

Permanent