Skip to content

This is where I put things I find useful that speed up my work with Machine Learning. Ever looked in your old projects to reuse those cool functions you created before? Well, this repo is designed to be a Python Library of functions I created in my previous project that can be reused. I also share some Notebooks Tutorials and Python Code Snippets.

License

Notifications You must be signed in to change notification settings

gmihaila/ml_things

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Machine Learning Things

ml_things Generic badge License Generic badge Generic badge pages-build-deployment

Machine Learning Things is a lightweight python library that contains functions and code snippets that I use in my everyday research with Machine Learning, Deep Learning, NLP.

I created this repo because I was tired of always looking up same code from older projects and I wanted to gain some experience in building a Python library. By making this available to everyone it gives me easy access to code I use frequently and it can help others in their machine learning work. If you find any bugs or something doesn't make sense please feel free to open an issue.

That is not all! This library also contains Python code snippets and notebooks that speed up my Machine Learning workflow.

Note:

  • Update: Feb 5, 2022 Thank you all again for your support and kindness! This package is available on pypi now! pip install ml-things
  • Update: July 16, 2021 Thank you all for your support and kindness! As I promissed I will move this repo to pip install modules.
  • If I reach 100 stars I will release the first official version and add it to the pip install modules!

Table of contents


ML_things

Installation

This repo is tested with Python 3.6+.

It's always good practice to install ml_things in a virtual environment. If you guidance on using Python's virtual environments you can check out the user guide here.

You can install ml_things with pip from GitHub:

pip install git+https://github.com/gmihaila/ml_things

Or from pypi:

pip install ml-things

Functions

All function implemented in the ml_things module.

Array Functions

Array manipulation related function that can be useful when working with machine learning.

pad_array [source]

Pad variable length array to a fixed numpy array. It can handle single arrays [1,2,3] or nested arrays [[1,2],[3]].

By default it will padd zeros to the maximum length of row detected:

>>> from ml_things import pad_array
>>> pad_array(variable_length_array=[[1,2],[3],[4,5,6]])
array([[1., 2., 0.],
       [3., 0., 0.],
       [4., 5., 6.]])

It can also pad to a custom size and with cusotm values:

>>> pad_array(variable_length_array=[[1,2],[3],[4,5,6]], fixed_length=5, pad_value=99)
array([[ 1.,  2., 99., 99., 99.],
       [ 3., 99., 99., 99., 99.],
       [ 4.,  5.,  6., 99., 99.]])

batch_array [source]

Split a list into batches/chunks. Last batch size is remaining of list values. Note: This is also called chunking. I call it batches since I use it more in ML.

The last batch will be the reamining values:

>>> from ml_things import batch_array
>>> batch_array(list_values=[1,2,3,4,5,6,7,8,8,9,8,6,5,4,6], batch_size=4)
[[1, 2, 3, 4], [5, 6, 7, 8], [8, 9, 8, 6], [5, 4, 6]]

Plot Functions

Plot related function that can be useful when working with machine learning.

plot_array [source]

Create plot from a single array of values.

All arguments are optimized for quick plots. Change the magnify arguments to vary the size of the plot:

>>> from ml_things import plot_array
>>> plot_array([1,3,5,3,7,5,8,10], path='plot_array.png', magnify=0.1, use_title='A Random Plot', start_step=0.3, step_size=0.1, points_values=True, use_ylabel='Thid', use_xlabel='This')

plot_array

plot_dict [source]

Create plot from a single array of values.

All arguments are optimized for quick plots. Change the magnify arguments to vary the size of the plot:

>>> from ml_things import plot_dict
>>> plot_dict({'train_acc':[1,3,5,3,7,5,8,10],
                'valid_acc':[4,8,9]}, use_linestyles=['-', '--'], magnify=0.1, 
                start_step=0.3, step_size=0.1,path='plot_dict.png', points_values=[True, False], use_title='Title')

plot_dict

plot_confusion_matrix [source]

This function prints and plots the confusion matrix. Normalization can be applied by setting normalize=True.

All arguments are optimized for quick plots. Change the magnify arguments to vary the size of the plot:

>>> from ml_things import plot_confusion_matrix
>>> plot_confusion_matrix(y_true=[1,0,1,1,0,1], y_pred=[0,1,1,1,0,1], magnify=0.1, use_title='My Confusion Matrix', path='plot_confusion_matrix.png');
Confusion matrix, without normalization
array([[1, 1],
       [1, 3]])

plot_confusion_matrix

Text Functions

Text related function that can be useful when working with machine learning.

clean_text [source]

Clean text using various techniques:

>>> from ml_things import clean_text
>>> clean_text("ThIs is $$$%.  \t\t\n \\ so dirtyyy$$ text :'(.   omg!!!", full_clean=True)
'this is so dirtyyy text omg'

Web Related

Web related function that can be useful when working with machine learning.

download_from [source]

Download file from url. It will return the path of the downloaded file:

>>> from ml_things import  download_from
>>> download_from(url='https://raw.githubusercontent.com/gmihaila/ml_things/master/setup.py', path='.')
'./setup.py'

Snippets

This is a very large variety of Python snippets without a certain theme. I put them in the most frequently used ones while keeping a logical order. I like to have them as simple and as efficient as possible.

Name Description
Read FIle One liner to read any file.
Write File One liner to write a string to a file.
Debug Start debugging after this line.
Pip Install GitHub Install library directly from GitHub using pip.
Parse Argument Parse arguments given when running a .py file.
Doctest How to run a simple unittesc using function documentaiton. Useful when need to do unittest inside notebook.
Fix Text Since text data is always messy, I always use it. It is great in fixing any bad Unicode.
Current Date How to get current date in Python. I use this when need to name log files.
Current Time Get current time in Python.
Remove Punctuation The fastest way to remove punctuation in Python3.
PyTorch-Dataset Code sample on how to create a PyTorch Dataset.
PyTorch-Device How to setup device in PyTorch to detect if GPU is available.

Comments

These are a few snippets of how I like to comment my code. I saw a lot of different ways of how people comment their code. One thing is for sure: any comment is better than no comment.

I try to follow as much as I can the PEP 8 β€” the Style Guide for Python Code.

When I comment a function or class:

# required import for variables type declaration
from typing import List, Optional, Tuple, Dict

def my_function(function_argument: str, another_argument: Optional[List[int]] = None,
                another_argument_: bool = True) -> Dict[str, int]
       r"""Function/Class main comment. 

       More details with enough spacing to make it easy to follow.

       Arguments:
       
              function_argument (:obj:`str`):
                     A function argument description.
                     
              another_argument (:obj:`List[int]`, `optional`):
                     This argument is optional and it will have a None value attributed inside the function.
                     
              another_argument_ (:obj:`bool`, `optional`, defaults to :obj:`True`):
                     This argument is optional and it has a default value.
                     The variable name has `_` to avoid conflict with similar name.
                     
       Returns:
       
              :obj:`Dict[str: int]`: The function returns a dicitonary with string keys and int values.
                     A class will not have a return of course.

       """
       
       # make sure we keep out promise and return the variable type we described.
       return {'argument': function_argument}

Notebooks Tutorials

This is where I keep notebooks of some previous projects which I turnned them into small tutorials. A lot of times I use them as basis for starting a new project.

All of the notebooks are in Google Colab. Never heard of Google Colab? πŸ™€ You have to check out the Overview of Colaboratory, Introduction to Colab and Python and what I think is a great medium article about it to configure Google Colab Like a Pro.

If you check the /ml_things/notebooks/ a lot of them are not listed here because they are not in a 'polished' form yet. These are the notebooks that are good enough to share with everyone:

Name Description Links
πŸ‡ Better Batches with PyTorchText BucketIterator How to use PyTorchText BucketIterator to sort text data for better batching. Open In Colab Generic badge Generic badge Generic badge Generic badge
🐢 Pretrain Transformers Models in PyTorch using Hugging Face Transformers Pretrain 67 transformers models on your custom dataset. Open In Colab Generic badge Generic badge Generic badge Generic badge
🎻 Fine-tune Transformers in PyTorch using Hugging Face Transformers Complete tutorial on how to fine-tune 73 transformer models for text classification β€” no code changes necessary! Open In Colab Generic badge Generic badge Generic badge Generic badge
βš™οΈ Bert Inner Workings in PyTorch using Hugging Face Transformers Complete tutorial on how an input flows through Bert. Open In Colab Generic badge Generic badge Generic badge Generic badge
🎱 GPT2 For Text Classification using Hugging Face πŸ€— Transformers Complete tutorial on how to use GPT2 for text classification. Open In Colab Generic badge Generic badge Generic badge Generic badge

Final Note

Thank you for checking out my repo. I am a perfectionist so I will do a lot of changes when it comes to small details.

If you see something wrong please let me know by opening an issue on my ml_things GitHub repository!

A lot of tutorials out there are mostly a one-time thing and are not being maintained. I plan on keeping my tutorials up to date as much as I can.


Contact 🎣

🦊 GitHub: gmihaila

🌐 Website: gmihaila.github.io

πŸ‘” LinkedIn: mihailageorge

πŸ“¬ Email: [email protected]


About

This is where I put things I find useful that speed up my work with Machine Learning. Ever looked in your old projects to reuse those cool functions you created before? Well, this repo is designed to be a Python Library of functions I created in my previous project that can be reused. I also share some Notebooks Tutorials and Python Code Snippets.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages