chore(examples): Adding Small Language Models fine-tuning example

[jesuino]

I have been using this notebook for other projects and I think it is interesting to have it here for showing Kale capabilities. It makes use of:

• Base Images: We must use Torch base image or this will never work well due high disk usage
• New features: HTML report is disabled for some nodes and it also uses the last line as output param feature
• Modern Libraries: It uses Hugging Faces SFT Trainer library

Furthermore, I do believe this is an example that can be used in actual production system or for learning and experimentation. Fine tuning small language models is being done by small to medium companies and bt practitioners who want to explore the possibilities of local LLMs execution.

Finally I do plan to expand this to use PVC and the new feature of embedding notebooks because I do use other workflow to convert the resulting model to GGUF and it could use the new Multi-Notebook composition feature.

How to test this:

Be aware that it requires a few morehardware resources for the minikube installation and also you need to setup the pypi production URL to allow overriding packages on the pod. These instructions are part of the example:

This notebook can run as a Kubeflow Pipeline, but it is required to set the production PyPi repository with flag --break-system-packages. Hence before running Jupyter Lab with Kale or using the kale command the env var KALE_PYPI_PROD_URL should be correctly configured:

export KALE_PYPI_PROD_URL="https://pypi.org/simple --break-system-packages"

The kubernetes installation running the pipeline should also have at least 20gb of free space and 16 gb of RAM (32 gb recommended). For a local minikube installation the following command can be used:

$ minikube start --disk-size=60g   --memory=16384

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from jesuino. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[adrielparedes]

@jesuino I see a lot of information from the runs in the file, is that correct?

[jesuino]

@adrielparedes thanks for letting me , I just cleaned the cells outputs

[hmtosi]

/lgtm
Hi @jesuino thank you for this work! Overall the code looks good to me. I am able to start and run the pipeline, although it dies on the train_model_step because my cluster runs out of space (so I haven't been able to see the outcome metrics).

I just have a few notes on the machine learning strategy:

• Overall the model is likely to need a larger data set to really be able to train and produce an accurate performance estimate, but I think this is a great size data set for initial exploration (it allows users to see how data variability will affect model performance)
• The Sample Data basically has three Strata - it has prompts for run_notebook, list_pipelines, and list_runs, which are mutually exclusive categories
• In a real-world example, it would be important to make sure the strata are evenly distributed across the train and test sets (ie. the same proportion of each stratum is represented in each data set). It is good that you shuffle the data before splitting them into train/test, but it could still be the case that one of the strata is over- or under-represented in the train set. This would cause over- or under-fitting on that stratum, which will cause the model to perform unequally on the all the strata in the test set.
• I also notice that the run_notebook data include parameters, while the other two strata do not include parameters. If you think about this like a data table, this is essentially an extra column of data that the other two strata do not have, which the model needs to take into consideration. This creates a lot more variability within the run_notebook stratum, which may make it more difficult for the model to predict on these prompts than on the others. This could lead the model to perform more poorly on run_notebook than on the others.

Overall, I think this is a wonderful toy example, and a great addition to the Kale repertoire.

[ada333]

thanks @jesuino for creating this! it is looking great, I have not tried to run it, I just went through the code. I have few comments that could improve the example.

[ada333]

could also be training data from PVC 😛 but it is not necessary and you would have to do more changes in the notebook for it to work

[jesuino]

Right now I don't see how to upload data from the workbench to the PVC - but I know it is possible to share the same PVC as the notebook. I would do it when we figure it out a way to run the pipeline and also add files to the PVC!

[ada333]

I would add this to separate json file - added in the gen_ai folder and only reference it here, so the notebook is easier to read

[jesuino]

I do agree, but the problem is during runtime, the pipeline won't have access to the file

[ada333]

I believe this step does not need cuda, and could just have pytorch image - image:docker.io/pytorch/pytorch:2.13.0-cpu
would save few GBs I think

[jesuino]

I do agree with you, but I didn't find CPU images on the official pytorch images repo, I found this intel one, but it is outdated, there's no image for pytorch 2.13: https://hub.docker.com/r/intel/pytorch

[google-oss-prow]

New changes are detected. LGTM label has been removed.

[jesuino]

Hello @ada333

The example was updated to use PVC. Please let me know your thoughts!