Pipeline orchestration
Separate preprocessing, model training and evaluation into defined stages managed through a DVC pipeline.
DVC PIPELINETO MOMNA'S ENGINEERING WORLD
End-to-End MLOps Pipeline with DVC, MLflow & DagsHub
Building a reproducible machine learning workflow with data preprocessing, Random Forest training, evaluation and experiment tracking.
Separate preprocessing, model training and evaluation into defined stages managed through a DVC pipeline.
DVC PIPELINETrack training parameters, evaluation metrics and related artifacts with MLflow to make experiments easier to inspect.
MLFLOWUse DVC to manage pipeline definitions, dependencies and versioned project artifacts for repeatable execution.
VERSION CONTROLUse DagsHub with the project workflow to support remote data and artifact storage and experiment collaboration.
DAGSHUBA machine learning workflow involves more than fitting a model. Data preparation, experiment configuration, artifacts and evaluation must be organized into a repeatable process. This project structures those steps into a pipeline using DVC, MLflow and DagsHub.
Build a reproducible machine learning pipeline that connects data preprocessing, Random Forest model training and evaluation, while tracking experiments and managing versioned artifacts.
03 / ARCHITECTURE & SYSTEM DESIGN
The project organizes machine learning into three defined DVC stages: preprocessing, training and evaluation. DVC manages pipeline execution and versioned artifacts, while MLflow records experiment information. DagsHub supports the project's remote workflow.
Raw dataset, project parameters and pipeline definitions.
The preprocessing stage prepares data for model training.
Train a Random Forest classifier and perform configured hyperparameter tuning.
Load the saved model and calculate evaluation accuracy.
DVC defines the preprocessing, training and evaluation stages and manages versioned pipeline artifacts. MLflow tracks experiment parameters, metrics and artifacts. DagsHub provides the project's remote collaboration and storage environment.
04 / IMPLEMENTATION & CORE COMPONENTS
This project separates data preprocessing, model training and evaluation into defined stages. DVC organizes the pipeline and its dependencies, MLflow records experiment information, and DagsHub supports remote collaboration around the workflow.
ENGINEERING PRINCIPLE
Keep preprocessing, training and evaluation independently identifiable, with pipeline dependencies and parameters defined in configuration. Track experiment results separately from the pipeline's stage order. This project demonstrates a reproducible training workflow—not a deployed production prediction service.
The preprocessing stage prepares the dataset for the model-training workflow and produces the processed data used by later stages.
Source:src/preprocess.pyTrains the classifier using the project's training workflow and includes GridSearchCV hyperparameter tuning.
Source:src/train.pyLoads the saved model and calculates accuracy as part of the evaluation stage.
Source:src/evaluate.pyDefines preprocessing, training and evaluation stages, their dependencies and the pipeline configuration used for reproduction.
Source:dvc.yamlRecords experiment parameters, evaluation metrics and associated artifacts so model-training runs can be inspected.
Role: Experiment recordsProvides the project's remote collaboration and storage environment alongside its DVC and MLflow workflow.
Role: Remote collaborationThe MLOps workflow separates data preparation, model training and evaluation into defined stages. DVC coordinates pipeline execution, while MLflow records experiment information and DagsHub supports the project's remote workflow.
EXECUTION PRINCIPLE
A reproducible pipeline depends on explicit stages, configuration and tracked artifacts. Run the defined workflow, inspect each stage's output, and use the recorded evaluation rather than assuming model quality from successful execution alone.
Begin with the raw dataset and the project configuration. Parameters are defined separately so pipeline settings can be inspected and adjusted.
data/raw/ · params.yamlThe preprocessing script prepares the input data and produces the processed dataset used by the subsequent training stage.
src/preprocess.pyThe training script fits a Random Forest classifier and performs GridSearchCV hyperparameter tuning. The workflow logs experiment parameters, metrics and related artifacts with MLflow.
src/train.pyThe evaluation script loads the saved model and calculates accuracy. The resulting metric should be read from the actual run output rather than assumed in advance.
src/evaluate.pydvc repro
DVC defines and runs the pipeline stages and manages versioned data and model artifacts. MLflow records experiment parameters, metrics and artifacts. DagsHub provides the remote collaboration and storage environment used by the project.
Review the configured stages and their dependencies in dvc.yaml.
Inspect the lock file to understand recorded pipeline state and tracked dependencies.
OPEN DVC.LOCK ↗Values must be read from actual execution output.
Review how the project evaluates the trained model. No unverified accuracy score is displayed here.
OPEN EVALUATE.PY ↗Inspect the training code for MLflow logging and model-training configuration. Confirm actual recorded runs in the experiment interface.
OPEN TRAIN.PY ↗Evidence boundary: these links lead to repository source files. They demonstrate implementation structure, not independently verified successful execution. Add real run screenshots when available.
An MLOps pipeline should make its stages, training configuration and evaluation process inspectable. This section connects the implementation to the evidence needed to verify how the workflow behaves.
Run dvc repro in the configured project environment and inspect the actual command output. Do not label a run successful without observing its result.
Check the MLflow tracking interface for recorded parameters, metrics and artifacts associated with an actual training run.
Use the evaluation output to report the real metric and its value. Explain what was measured instead of presenting a decorative metric as a result.
Review the tracked configuration, dependencies and outputs, then verify whether the workflow can be reproduced in the intended environment.
What counts as proof? A genuine execution log, experiment record or evaluation output that can be traced back to this repository.
A reproducible ML workflow depends on more than training code. Credentials, pipeline dependencies, experiment records and evaluation outputs all need to be handled carefully. This review distinguishes documented project capabilities from operational safeguards that still require verification.
Versioned artifacts and tracked experiments improve traceability, but do not automatically guarantee security, model quality or production reliability.
The project source contains tracking credential assignments. Do not treat source-code credentials as a secure secret management strategy. Use appropriately configured environment variables or a secrets manager, and rotate any exposed token.
DVC configuration and lock-file metadata help describe pipeline stages, dependencies and outputs. Inspect these files when investigating unexpected results or changes between runs.
MLflow records configured experiment information such as parameters, metrics and artifacts. These records support investigation, but their completeness depends on what the implementation logs.
Inspect command output, dependencies, parameters and the relevant pipeline stage before rerunning. Check evaluation output separately; do not infer model quality from a successful command alone.
This project documents an MLOps pipeline. Do not claim automated rollback, production monitoring, a completed security audit or verified access controls without separate implementation evidence.
MLflow supports experiment tracking, while params.yaml keeps configurable pipeline
values in a dedicated file. DagsHub provides the
project's remote collaboration and tracking context.
Defines pipeline stages and tracks pipeline state.
Requires maintained stage definitions and dependencies.
Separates supported configuration values from source code.
Configuration changes still require careful validation.
Organizes experiment-tracking information for model development.
Useful comparisons depend on consistent logging and run records.
Connects the project with remote data-versioning and tracking workflows.
Remote operations depend on correct configuration and access.
Reproducibility is a workflow design goal, not an automatic guarantee. Data, parameters, dependencies and experiment records must remain aligned for results to be meaningfully reproduced and compared.
This project combines pipeline orchestration, configurable execution and experiment tracking. Each component serves a distinct purpose in making the model-development workflow easier to organize, inspect and maintain.
DVC manages pipeline structure while MLflow focuses on experiment tracking. They complement each other rather than replacing one another.
Versioned pipeline definitions and externalized parameters improve traceability, but add configuration and dependency management responsibilities.
Experiment tracking supports model-development analysis. Production serving, deployment automation and monitoring remain separate capabilities—not assumed deliverables.
This project brings data versioning, pipeline orchestration, model training, evaluation and experiment tracking into one structured MLOps workflow. Its value lies in the foundations it establishes—not in unverified performance claims.
Separate experiment tracking from pipeline orchestration so model-development activity is easier to inspect.
Versioned data references, consistent parameters and maintained dependencies all contribute to repeatable work.
Assessing a model is a distinct step from serving it in production. Deployment and production monitoring remain future extensions of this project.
The repository defines DVC pipeline stages for preprocessing, training and evaluation, supported by pipeline configuration and tracked state files.
dvc.yaml dvc.lock params.yamlMLflow tracking and DagsHub integration form part of the project's approach to organizing model-development activity and remote project resources.
Evidence-first reporting No accuracy score, runtime improvement or successful end-to-end run is claimed here without verified execution evidence.
Review the project source, pipeline definitions, configuration and supporting implementation files.
github.com/momna-shahid17/mlops-dvc-mlflow-pipelineValidate reproducibility with a clean end-to-end pipeline run.
Capture verified evaluation metrics and experiment comparisons.
Explore model serving, deployment automation and production monitoring.
The source repository is the starting point for reviewing this project’s pipeline structure, model-development code and configuration. The next stage is to validate the workflow through repeatable runs and extend it where the requirements demand.
Start with the pipeline definition and configuration files to understand how the workflow is organized.
Use actual pipeline output, evaluation results and experiment records to support technical conclusions.
Treat deployment, automated delivery and production monitoring as future work unless implemented and verified.