Skip to content
View muk0644's full-sized avatar

Highlights

  • Pro

Block or report muk0644

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
muk0644/README.md

👋 Hi, I'm Muhammad Shariq Khan

AI Engineer | Architecting End-to-End GenAI Systems & Agentic AI
Specializing in Multimodal LLMs, VLMs, and Scalable MLOps/Inference Pipelines

PortfolioLinkedInEmailGitHub


💡 About Me

I am an AI Engineer bridging the gap between research and production. I don't just train models; I architect end-to-end AI systems. My expertise spans fine-tuning Multimodal Transformers (VLMs) to deploying scalable inference pipelines using FastAPI, Docker, and Agentic Workflows.

🎓 M.Eng. in AI Engineering for Autonomous Systems
💼 Experience in LLMs, Data Engineering, Predictive Analytics, MLOps, LLMOps, GitHub Actions, Docker, CI/CD
🌱 Always learning and exploring new technologies in AI and ML
📍 Based in Germany | Fluent in English & German (C1)


🚀 Featured Projects

#ProjectDescriptionTech Stack
1️⃣SmolAgents AI AssistantFully functional, tool-augmented AI assistant with autonomous tool-calling (web research, weather, image generation)SmolAgents, Qwen2.5-Coder, Gemini 2.5 Flash, Streamlit, Docker, GitHub Actions, Chroma DB
2️⃣AI-Driven Predictive Maintenance for Aircraft EngineEnd-to-end predictive system for aircraft turbofan engines with 99.15% RUL prediction accuracy using hybrid CNN-LSTMPython, TensorFlow/Keras, CNN-LSTM, Pandas
3️⃣LLM-based Agent for Driver Fatigue DetectionEmbodied AI agent using LLMs for real-time driver safety with fine-tuned LLaMA 2 and multimodal sensor fusionROS 2, Meta LLaMA 2, LoRA/PEFT, RAG (FAISS), OpenCV, MediaPipe, CARLA
4️⃣Robust Vehicle State EstimationHybrid deep learning framework combining RNN, Transformer, and PINNs with Kalman Filter for vehicle state estimationPython, PyTorch, GRU, Transformers, PINNs
5️⃣Smart City Traffic ControlDeep Reinforcement Learning system optimizing traffic light phases for congestion reductionPython, PyTorch, Deep Q-Network (DQN), SUMO Simulator, NumPy
6️⃣Urban Traffic Analysis & PredictionEnd-to-end ML pipeline reducing MSE by 61% with K-Means clustering for traffic behavior identificationPyTorch, Scikit-learn, Pandas, K-Means Clustering, Folium
7️⃣E2E DriveAI: ROS2 Modular FrameworkComplete self-driving car system with sensor fusion (Camera + LiDAR) and CUDA-optimized inferenceROS 2, PyTorch, Transformers, CUDA, ResNet-18, PointPillars, Open3D
8️⃣California Housing Price PredictorComprehensive ML pipeline for house price prediction with automated feature engineering and Random ForestScikit-Learn, Random Forest, Pandas, NumPy, Joblib
9️⃣Mini-CNN Framework: C++ Inference EngineLightweight CNN framework built from scratch in C++17 with INT8 quantization reducing memory by 75%C++17, STL, Make, INT8 Quantization, Im2col
🔟Autonomous Agent Training using Q-LearningSelf-learning agent for complex grid environments with custom "ShariqQuest" environmentPython, PyTorch, Gymnasium, Pygame
1️⃣1️⃣3DGazeNet Extended Gaze EstimationNon-intrusive Driver Monitoring System with 90.5% gaze classification accuracyResNet-18, Geometric Computer Vision, Deep Learning
1️⃣2️⃣LangChain-MongoDB Chat AssistantIntelligent chat application with persistent message history and multi-user supportStreamlit, LangChain, MongoDB Atlas, Google Generative AI (Gemini)
1️⃣3️⃣Vision-LLM Traffic Analysis & LoRA Fine-TuningFine-tuned Qwen2-VL-7B for thermal camera traffic counting with 92.76% accuracy using Sculptor MethodPyTorch, Qwen2-VL, LoRA (PEFT), BitsAndBytes, Docker

🛠️ Tech Stack

🧠 AI & GenAI Stack

PythonPyTorchTensorFlowCUDAScikit-LearnKerasHuggingFaceLangChainSmolAgentsOpenCVGymnasiumTool_CallingPygame

⚙️ Engineering & MLOps

FastAPIDockerLinuxROS2CI%2FCDAWSGCPStreamlitHugging Face SpacesJupyterVS CodeBashGit

📊 Data & Analytics

NumPyPandasMatplotlibSeabornPower BIChromaDBMongoDBPostgreSQLMySQLSQLite

🛠️ IDEs & Development Tools

GitHub CopilotCursorGoogle Antigravity


🧠 Neural Network Architectures & AI Models

Deep Learning Architectures:

  • 🔷 CNNs (Convolutional Neural Networks) - Image processing & computer vision
  • 🔁 RNNs (Recurrent Neural Networks) - Sequential data & time series
  • 🔄 LSTMs (Long Short-Term Memory) - Long-term dependencies & memory
  • Transformers & Attention Mechanisms - State-of-the-art sequence modeling

Advanced AI Models:

  • 🤖 LLMs (Large Language Models) - Text generation & understanding
  • 👁️ VLMs (Vision-Language Models) - Multimodal understanding
  • 🎯 RL Agents (Q-Learning, DQN) - Decision-making systems
  • 🔍 Agentic AI - Multi-step reasoning and autonomous agent systems

🎯 Areas of Expertise

skills= {
"AI & ML": ["Deep Learning", "Neural Networks", "LLMs", "Computer Vision", "NLP"],
"Frameworks": ["PyTorch", "TensorFlow", "Keras", "Transformers", "LangChain", "SmolAgents", "OpenCV"],
"Data Science": ["Data Analysis", "Statistical Modeling", "Feature Engineering"],
"MLOps": ["Model Training", "Model Validation", "Model Evaluation", "Model Deployment", "Optimization"],
"Tools": ["Git", "Docker", "Linux", "Jupyter", "Power BI", "ROS2"]
}

🎯 What I Actually Do

classAIEngineer:
def__init__(self):
self.name="Muhammad Shariq Khan"self.role="AI Engineer"self.goal="Building Systems, Not Just Models"defdaily_workflow(self):
return [
"Fine-tune Multimodal Transformers (VLMs)",
"Design scalable inference pipelines",
"Deploy production-ready AI systems",
"Implement Agentic Workflows with Tool Calling"
]

� Featured Work

🤖 Multimodal LLM Fine-tuning

Specialized in fine-tuning large language models using parameter-efficient techniques (LoRA/PEFT) for computer vision tasks, achieving significant performance improvements over baseline models.

1️⃣ SmolAgents AI Assistant

🎯 Fully functional, tool-augmented AI assistant with autonomous tool-calling (web research, weather, image generation) 🛠️ Tech: SmolAgents, Qwen2.5-Coder, Gemini 2.5 Flash, Streamlit, Docker, GitHub Actions, Chroma DB

2️⃣ AI-Driven Predictive Maintenance for Aircraft Engine

🎯 End-to-end predictive system for aircraft turbofan engines using NASA C-MAPSS sensor data 🎯 Predicts Remaining Useful Life (RUL) with 99.15% accuracy using hybrid CNN-LSTM model 🛠️ Tech: Python, TensorFlow/Keras, CNN-LSTM, Pandas

3️⃣ LLM-based Agent for Driver Fatigue Detection

🎯 Embodied AI agent using LLMs as reasoning engine for real-time driver fatigue detection 🎯 Fine-tuned LLaMA 2 for edge deployment with multimodal sensor fusion in CARLA simulator 🛠️ Tech: ROS 2, Meta LLaMA 2, LoRA/PEFT, RAG (FAISS), OpenCV, MediaPipe, CARLA

4️⃣ Robust Vehicle State Estimation

🎯 Hybrid deep learning framework for vehicle state estimation (position, velocity, orientation) 🎯 Combines RNN, Transformer, and Physics-Informed Neural Networks (PINNs) with Kalman Filter fusion 🛠️ Tech: Python, PyTorch, GRU, Transformers, PINNs

5️⃣ Smart City Traffic Control

🎯 Deep Reinforcement Learning system optimizing traffic light phases for congestion reduction 🎯 DQN with Experience Replay integrated with SUMO traffic simulator 🛠️ Tech: Python, PyTorch, Deep Q-Network (DQN), SUMO Simulator, NumPy

6️⃣ Urban Traffic Analysis & Prediction

🎯 End-to-end ML pipeline for large-scale traffic flow analysis using UTD19 dataset 🎯 Custom Neural Network reducing MSE by 61% with K-Means clustering for behavior identification 🛠️ Tech: PyTorch, Scikit-learn, Pandas, K-Means Clustering, Folium

7️⃣ E2E DriveAI: ROS2 Modular Framework

🎯 Self-driving car system using deep learning for steering and speed control from sensor data 🎯 Sensor fusion of Camera + LiDAR with custom CUDA-optimized processing for high-speed inference 🛠️ Tech: ROS 2, PyTorch, Transformers, CUDA, ResNet-18, PointPillars, Open3D

8️⃣ California Housing Price Predictor

🎯 Comprehensive ML pipeline for house price prediction with automated feature engineering 🎯 Random Forest achieving low error rates with stratified sampling for data reliability 🛠️ Tech: Scikit-Learn, Random Forest, Pandas, NumPy, Joblib

9️⃣ Mini-CNN Framework: C++ Inference Engine

🎯 Lightweight CNN framework built from scratch using C++17 without external ML libraries 🎯 LeNet-5 architecture with INT8 quantization reducing memory by 75%, optimized with Im2col algorithm 🛠️ Tech: C++17, STL, Make, INT8 Quantization, Im2col

🔟 Autonomous Agent Training using Q-Learning

🎯 Self-learning autonomous agent for complex grid environments using Standard Q-Learning and DQN 🎯 Built custom "ShariqQuest" environment with comprehensive hyperparameter tuning 🛠️ Tech: Python, PyTorch, Gymnasium, Pygame

1️⃣1️⃣ 3DGazeNet Extended Gaze Estimation

🎯 Non-intrusive Driver Monitoring System (DMS) for assessing driver alertness 🎯 Extended 3DGazeNet with eye-behavior analysis and 90.5% gaze classification accuracy 🛠️ Tech: ResNet-18, Geometric Computer Vision, Deep Learning

1️⃣2️⃣ LangChain-MongoDB Chat Assistant

🎯 Intelligent chat application with persistent message history across sessions. 🎯 Features multi-user support and real-time responses using Gemini 2.5 Flash. 🛠️ Tech: Streamlit, LangChain, MongoDB Atlas, Google Generative AI (Gemini), Python-dotenv

1️⃣3️⃣ Vision-LLM Traffic Analysis & LoRA Fine-Tuning

🎯 Fine-tuned Qwen2-VL-7B for thermal camera traffic object counting with 92.76% accuracy. 🎯 Implemented the Sculptor Method for efficient 4-bit LoRA training, reducing VRAM by 43%. 🛠️ Tech: PyTorch, Qwen2-VL, LoRA (PEFT), BitsAndBytes, UrbanIng-V2X Dataset, Docker


💼 Experience Highlights

AI Research & Development - Working with cutting-edge LLMs and multimodal models
Data Engineering - Building robust pipelines for production ML systems
Predictive Analytics - Condition-based maintenance and reliability optimization
Automation - Creating automated reporting systems and workflows


📚 Continuous Learning

I'm passionate about staying at the forefront of AI technology:

  • 📖 Exploring latest research in LLMs and transformer architectures
  • 🧪 Experimenting with new ML frameworks and tools
  • 🤝 Contributing to open-source projects
  • 💡 Sharing knowledge and learning from the community

🤝 Let's Connect!

I'm always interested in:

  • 💬 Discussing AI, ML, and Data Science projects
  • 🤝 Collaborating on innovative solutions
  • 📚 Sharing knowledge and experiences
  • 🌟 Exploring new opportunities

Reach out to me:


GitHub Stats

Top Languages


Profile Views

Pinned Loading

  1. GazeNet-Extended-Gaze-EstimationGazeNet-Extended-Gaze-EstimationPublic

    Enhanced gaze estimation with drowsiness, blink, yawn detection and depth estimation. An extended implementation of 3DGazeNet with additional eye-behavior analysis features for real-world applicati…

    Python 1 1

  2. AI-Driven-Predictive-Maintenance-for-Aircraft-Engine-using-ML-and-DLAI-Driven-Predictive-Maintenance-for-Aircraft-Engine-using-ML-and-DLPublic

    End-to-end AI-driven predictive maintenance system for aircraft turbofan engines using NASA C-MAPSS sensor data. Hybrid CNN-LSTM model achieves 99.15% accuracy (RMSE: 5.27 cycles) for Remaining Use…

    Python

  3. Autonomous-Vehicle-State-Estimation-Physics-Informed-Deep-LearningAutonomous-Vehicle-State-Estimation-Physics-Informed-Deep-LearningPublic

    High-precision vehicle localization using machine learning and physics-based models. Achieves <2m positioning error over 60s using only low-cost OBD sensors. Compares RNN, GRU, LSTM, and Transforme…

    Jupyter Notebook

  4. langchain-mongodb-chatlangchain-mongodb-chatPublic

    Simple AI-powered chat application using Streamlit, LangChain, and MongoDB for persistent message history with Gemini 2.5 Flash

    Python

  5. LLM-based-agent-for-Driver-Fatigue-detection-and-mitigationLLM-based-agent-for-Driver-Fatigue-detection-and-mitigationPublic

    LLM-based intelligent system for real-time driver sleepiness detection and mitigation. Combines computer vision, FAISS vector database, and fine-tuned Llama models to monitor driver fatigue and pro…

    Python

  6. smolagents-ai-assistantsmolagents-ai-assistantPublic

    Intelligent Multi-tool AI agent powered by smolagents including Agentic RAG System powered by smolagents with LangChain integration using Qwen2.5-Coder-32B-Instruct & Gemini 2.5 Flash. Features: Ad…

    Python