Hi,
I'm Abhishek Shah

I'm a

Senior Data Scientist | AI/ML Engineer | Cloud Solutions Architect | Leading Enterprise AI Solutions | Expert in Generative AI & MLOps

Abhishek Shah
Abhishek Shah

About Me

Senior Data Scientist

I am a technology leader with 13+ years of experience in enterprise AI solutions, specializing in developing production-grade ML systems, LLM applications, and cloud-native AI solutions. I have a proven track record of leading cross-functional teams to deliver impactful business solutions through advanced analytics and machine learning. Outside of work, I enjoy hiking and spending time in the outdoors, finding inspiration in nature's complexity and beauty.

Experience

2024 - Present

Data Scientist - BMW

  • Led development and training of enterprise LLM platform serving 100,000+ users
  • Spearheaded the development of responsible AI use cases, ensuring alignment with business objectives to elevate business value and improve customer outcomes.
  • Collaborated with executive leadership and cross-functional teams to deploy transformative AI solutions across BMW’s global manufacturing network.
2019 - 2024

Data Scientist - Intel

  • Built a RAG pipeline with OpenAI & vector DB, automating SOW analysis for 45 engineers and saving 10 hours/week per engineer, cutting $100K/month in costs.
  • Developed an ML model for FOUP defect detection using Random Forest & PCA, optimized with OpenVINO, reducing scrap wafers and saving $250K yearly.
  • Implemented AI-driven quality control at Audi, analyzing 900 welding robots, enabling real-time inspection of 5M+ welds, and cutting labor costs by 50%.
  • Built Power BI dashboard with ADF to track 4,000+ pipelines, enabling early failure detection and saving the analytics team 1 hour/day in troubleshooting.
2017 - 2019

Data Analytics Process Engineer - Westlake Chemical

  • Utilized data analytics and statistical process control to design safeguards for brine recovery, reducing raw material usage by 20%.
  • Conducted feasibility analysis for a $4MM chlorine liquefaction system, selecting an environmentally friendly alternative that increased throughput by 15%.
  • Applied predictive modeling to redesign acid discharge pump systems, achieving a 25% reduction in acid consumption and improved efficiency.
  • Developed a safe work procedure for HCl burner sampling, saving $20,000 annually in lab consultation costs.
  • Led hazard analysis and operability studies using big data to assess process changes for safety and environmental impact.
  • Managed three plant shutdown projects, optimizing schedules with data analytics to reduce timelines by 5%.
2012 - 2017

Staff Data Analytics Process Engineer & Project Manager - Tate & Lyle

  • Managed $5MM+ capital projects, using data analytics for research, construction planning, and cost estimation.
  • Developed key performance indicators (KPIs) to monitor plant efficiency, optimizing production planning and downtime reduction.
  • Created a real-time dashboard using PI ProcessBook for enhanced process visibility and troubleshooting.
  • Led process hazard analysis and risk mitigation strategies to ensure safe equipment installation and operation.
  • Supervised up to 20 contractors during plant outages, optimizing resource allocation and minimizing costs through data analytics.

Education

2022 - 2025

M.S. Artificial Intelligence & Machine Learning - University of Michigan

Concentration: NLP | Relevant Courses: Advanced Deep Learning, ML Engineering, Cloud AI Systems

2018 - 2020

M.S in Engineering and Technology Management - Washington State University

Focus: Data-Driven Decision Making, Technology Strategy

2007 - 2012

Bachelor of Science in Chemical Engineering - University of Minnesota

Tech Stack

AI/ML

  • TensorFlow, PyTorch, HuggingFace, LangChain, OpenAI GPT, Claude

Cloud & DevOps

  • AWS SageMaker, Azure ML, Kubernetes, Docker, Airflow, Terraform

Data Engineering

  • Spark, Snowflake, Databricks, SQL, NoSQL, DataRobot, MLflow

Latest Projects

Local RAG with DeepSeek and Ollama

Local RAG with DeepSeek and Ollama

A Streamlit app for SEC filings (10-K, 10-Q, 8-K) using Ollama, DeepSeek, OpenAI embeddings, Pinecone, and local RAG for AI-driven search, analysis, and trend comparison.

Agentic Rag Financial Analysis AI Assistant

Agentic Rag Financial Analysis AI Assistant

This project implements AI agents using the Agno Agentic framework to fetch web search results and financial data and do Analysis using Agno Agent

Customer Support Intelligence with NLP and Gemini AI

Customer Support Intelligence with NLP and Gemini AI

An interactive Streamlit app leveraging NLP embeddings and Gemini AI to analyze, classify, and provide insights on customer support issues

Local RAG with DeepSeek and Ollama

RAG With Neo4J Knowledge Graph With OpenAI

This project implements a high-performance NLP pipeline for scientific document analysis, integrating a Neo4j knowledge graph for structured storage and retrieval. The Retrieval-Augmented Generation (RAG) system enables semantic search and contextual querying of scientific literature.

Legal Document Search Using NLP

Legal Document Search Using NLP

A Streamlit-based legal document search system using PySpark, BM25, and TF-IDF for efficient full-text retrieval. It preprocesses legal texts, builds an inverted index, and ranks results dynamically, enabling fast, AI-powered search and analysis.

Computer Vision Quality Control

Lyft Dynamic Pricing

A Streamlit-based Lyft Trip Cost Predictor using Linear Regression, scikit-learn, pandas, and joblib to estimate ride prices based on distance, time, and peak hours. đźš• Optimized for dynamic pricing analysis and real-time cost estimation.

Financial Doc Analyser

Financial Doc Analyser

AI-powered SEC filing analysis using OpenAI embeddings, Pinecone, and Streamlit for fast, structured search. Retrieve, process, and analyze 10-K, 10-Q, and 8-K filings with instant, context-aware insights.

Future Project 1

Credit Risk Modeling

A Streamlit-based loan default prediction app using XGBoost, Logistic Regression, Pandas, and NumPy. It analyzes loan characteristics and predicts default risk in real time based on user inputs.

Future Project 1

Fake Image/Video Detector Using Deep Learning and Gemini

The AI Fake Image & Video Detector is a powerful Streamlit-based application designed to identify whether images or videos are AI-generated or authentic. Utilizing advanced techniques, this tool helps detect synthetic media created by popular AI models such as DALL-E, Midjourney, Stable Diffusion, and others.

Future Project 2

Stock Market Market Analsyis

A stock market prediction system using RNN, LSTM, GRU, DNN, KNN, and Random Forest to forecast next-day closing prices. Built with Yahoo Finance, Pandas, Scikit-learn, and Matplotlib, it compares deep learning and machine learning models for accuracy.

Publications

June 11, 2023 Technical Article

Battle of Pathfinding Algorithms: A*, Branch & Bound, and Dijkstra’s Showdown in the 4 Knights…

Embark on an intriguing journey through the implementation of three search algorithms—A*, Branch and Bound (BnB), and Dijkstra—in the context of the 4 Knights problem.

Read Article
February 20, 2023 Case Study

Recommender System for Matching HealthCare Professionals with Jobs Using Cosine Similarity

Explore how a healthcare staffing company can offer a platform that allows healthcare facilities to easily find pre-qualified professionals using a recommender system based on cosine similarity.

Read Article
February 19, 2023 Technical Article

Implementation Of Generalized Linear Regression Model Using Moore-Penrose Inverse

This article presents an implementation of linear regression using a closed-form solution for a given dataset D.

Read Article
February 10, 2023 Philosophical Exploration

The Human Touch: Navigating the Intersection of Technology and Humanity

Delve into the philosophical exploration of the relationship between humanity and technology, emphasizing its growing importance in modern times.

Read Article
January 15, 2023 Technical Comparison

Comparison of AutoEncoders vs. Variational Autoencoders

Discuss the differences between AutoEncoders and Variational Autoencoders, highlighting their applications in representing data in lower-dimensional spaces.

Read Article
January 2, 2023 Technical Overview

YARN (Yet Another Resource Negotiator) Architecture

An overview of YARN, a resource management platform in Hadoop, detailing how it manages computing resources within a cluster.

Read Article
January 1, 2023 Theoretical Exploration

Computational Learning Theory In Machine Learning

An exploration of the theoretical foundations of machine learning algorithms, focusing on well-established concepts in computational learning theory.

Read Article
December 14, 2022 Research Application

Advancing Fusion Energy Research With Machine Learning

Discuss how machine learning is becoming an essential tool in fusion research, aiding scientists in making new discoveries and improving experimental outcomes.

Read Article
December 11, 2022 Technical Insight

Reshaping the Dataset For Neural Networks

Insights into how neural networks expect input data in specific shapes, and methods to reshape data to match these expectations for effective model training.

Read Article
December 9, 2022 Algorithm Comparison

DFS vs BFS Algorithms for Graph Traversal

A comparative analysis of Depth-First Search (DFS) and Breadth-First Search (BFS) algorithms, outlining their differences and use cases in graph traversal.

Read Article
December 9, 2022 Technical Overview

Learning Optimizers in Deep Learning

An overview of various optimizers used in deep learning, discussing their strengths, weaknesses, and applications in training neural networks.

Read Article
July 19, 2022 Technical Insight

Feature Engineering using Random Forest Classifier in Machine Learning

A classifier called Random Forest consists of a number of classifiers with a tree-like structure, identically distributed independent…

Read Article
July 19, 2022 Technical Overview

Decision Tree and Ensemble Learning Algorithms in Machine Learning

Classification and Regression Tree (CART) is a decision tree learning algorithm. It can be used for both classification and regression…

Read Article
July 19, 2022 Theoretical Exploration

Generalization Error in Machine Learning (Bias vs. Variance)

A fundamental goal of machine learning is generalization: the ability to draw inferences about unseen data from finite training examples…

Read Article
July 19, 2022 Theoretical Exploration

Overcoming overfitting a model in Machine Learning

A model that has learned the noise instead of the signal is considered “overfit” because it fits the training dataset but has a poor fit with new datasets. This is a common problem in machine learning which can be solved using a validation dataset. Generalization of a model to new data is ultimately what allows us to use machine learning algorithms every day to make predictions and classify data.

Read Article

My Podcasts

Published 3/4/25 Audio • 14:20

AI Powered SEC Analyzer

An analysis of how AI technologies are being used to analyze financial data and SEC filings.

Listen on Spotify
Published 10/9/24 Audio • 07:34

The Decade Ahead in AI

Exploring potential developments and impacts of artificial intelligence over the next ten years.

Listen on Spotify
Published 03/08/25 Audio • 18.41

Customer Support Intelligence with NLP and Gemini AI

An interactive Streamlit app leveraging NLP embeddings and Gemini AI to analyze, classify, and provide insights on customer support issues.

Listen on Spotify
Published 03/05/25 Audio • 18:09

NLP Pipeline with Neo4j Knowledge Graph for Scientific Literature

Exploring high-performance NLP pipeline that uses a Neo4j knowledge graph for the analysis of scientific documents built on a Retrieval-Augmented Generation (RAG) architecture

Listen on Spotify
Published 10/9/24 Audio • 08:03

Cracking the Protein Code: How AlphaFold Earned the 2024 Nobel Prize in Chemistry

Discussion on AlphaFold's revolutionary impact on protein structure prediction and its Nobel Prize recognition.

Listen on Spotify
Published 10/8/24 Audio • 09:32

A Survey of Dynamic Programming Algorithms

Exploring various dynamic programming algorithms and their applications in computer science.

Listen on Spotify
Published 10/8/24 Audio • 08:37

Retrieval Interleaved Generation (RIG) using LLM: What is It and How It Works?

Explaining the concept of Retrieval Interleaved Generation and its implementation with Large Language Models.

Listen on Spotify
Published 10/4/24 Audio • 10:05

Influence of a Large Language Model on Diagnostic Reasoning

Examining how large language models are impacting medical diagnosis and clinical decision-making.

Listen on Spotify
Published 10/4/24 Audio • 16:03

Scaling Laws for Neural Language Models

Discussing the mathematical principles that govern how neural language models improve with increased size and data.

Listen on Spotify
Published 10/4/24 Audio • 12:09

Latent Dirichlet Allocation

Exploring the popular topic modeling technique and its applications in natural language processing.

Listen on Spotify