intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What we'll study in data mining\n",
    "\n",
    "A famous problem is to write a program that can \"read\" the number that is contained in an image.\n",
    "\n",
    "<img src=\"img/mnist.png\" />\n",
    "\n",
    "It is hard to manually write a program to do this.\n",
    "In this class, we will study algorithms that learn how to perform this task automatically from data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the MNIST data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train.shape= (50000, 784)\n",
      "y_train.shape= (50000,)\n",
      "X_test.shape= (10000, 784)\n",
      "y_test.shape= (10000,)\n"
     ]
    }
   ],
   "source": [
    "# load MNIST dataset\n",
    "from sklearn.datasets import fetch_openml\n",
    "X, y = fetch_openml(\"mnist_784\", version=1, return_X_y=True)\n",
    "\n",
    "# create train/test split\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, train_size=50000, test_size=10000, random_state=0\n",
    ")\n",
    "\n",
    "# print debug info\n",
    "print('X_train.shape=', X_train.shape)\n",
    "print('y_train.shape=', y_train.shape)\n",
    "print('X_test.shape=', X_test.shape)\n",
    "print('y_test.shape=', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAD7CAYAAABDsImYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAF5UlEQVR4nO3dPWsUaxjH4ZmDINpEwReUQOwiaiPaaSmClZDKxlhbimIpiAh+B8EvYCFYSEBQsZA0ElBEAxLBQjQiaCP4xpzmwGnC80zcTTbZ/3W19zB7i/x4itmdtF3XdQ0w1v4Z9QLA2hM6BBA6BBA6BBA6BBA6BBA6BBA6BBA6BNjS98K2bddyD+Av9flyqxMdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAggdAmwZ9QLr6eDBg9Vrbt++XZw/fPiwOL9161Zx/vPnz+oOMGxOdAggdAggdAggdAggdAggdAggdAggdAjQdl3X9bqwbdd6l4EdP368OL979271HgcOHBhoh+fPnxfn169fr95jbm6uOP/169eqdmK89UnYiQ4BhA4BhA4BhA4BhA4BhA4BhA4BxurFEydOnCjOB31G3sexY8eK8/v371fvsbCwUJxfvny5OH/8+HH1M8jiRIcAQocAQocAQocAQocAQocAQocAY/UcfRh+/PhRnM/Ozhbnhw4dKs4vXLhQ3eHo0aPFee1Z/OvXr4vzmzdvVnd49uxZcX7kyJHivPbvvHPnTnWHp0+fVq+hHyc6BBA6BBA6BBA6BBA6BBA6BBA6BBir97pPT08X548ePareY//+/QPtUHt+/Pbt2+o9au9+P3Xq1Kp22ohu3LhRvebatWvrsMnm573uQNM0QocIQocAQocAQocAQocAQocAQocAY/XiicXFxeL88OHD1XucO3euOP/+/XtxPjc3V5wvLy9Xdzhz5kxxvnv37uL86tWrxfnMzEx1h61btxbne/furd6DjcOJDgGEDgGEDgGEDgGEDgGEDgGEDgHG6sUTDM/OnTuL8xcvXhTnk5OTxbkXTwyPF08ATdMIHSIIHQIIHQIIHQIIHQIIHQKM1e/RGZ6pqanifNeuXeu0CcPgRIcAQocAQocAQocAQocAQocAQocAnqOzotpvnHu+xoANwokOAYQOAYQOAYQOAYQOAYQOAYQOAYQOAXxhZhO6cuVKcf7x48fi/M+fP9XP2L59e3G+bdu26j1K9u3bV71mYmKiOP/27dtAOyRxokMAoUMAoUMAoUMAoUMAoUMAoUOAtuv5BoG2bdd6F/6zY8eO4nx+fr44n56eHuI2o7O0tFSc156jLy4uFufnz5+v7vD79+/qNaPWJ2EnOgQQOgQQOgQQOgQQOgQQOgQQOgTwHH0MXbp0qTifmZmp3uPkyZPDWmfNvHz5sjg/ffp0cV773f5m4Tk60DSN0CGC0CGA0CGA0CGA0CGA0CGA5+iB+vxf3rt3rzg/e/bsQDtcvHixes2TJ0+K88+fPxfnX758Wc1Km5bn6EDTNEKHCEKHAEKHAEKHAEKHAEKHAEKHAFtGvQDrr88XLF69elWcD/qFmQcPHlSvef/+/UCfwf+c6BBA6BBA6BBA6BBA6BBA6BBA6BDAc3RW9O7du1GvwBA50SGA0CGA0CGA0CGA0CGA0CGA0CGA5+is6M2bN6NegSFyokMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAoUMAf8CBFS0tLRXny8vLxfmePXuGuQ4DcqJDAKFDAKFDAKFDAKFDAKFDAKFDAM/RWdGHDx+K8/n5+eL869evxfmnT59WuxIDcKJDAKFDAKFDAKFDAKFDAKFDAKFDgLbruq7XhW271rsAf6FPwk50CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CCB0CND7Dzj0fD8FsAE50SGA0CGA0CGA0CGA0CGA0CGA0CGA0CGA0CHAvywy52oAEibzAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 300x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# visualize the data\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "image = X_train.iloc[0].to_numpy().reshape([28,28])\n",
    "fig = plt.figure(figsize=(3,3))\n",
    "plt.axis('off')\n",
    "plt.imshow(image, cmap='gray')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training a model\n",
    "\n",
    "Now that we have some data, we can train a model on the data.\n",
    "The following code trains and evaluates a Logistic Regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# train the model\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "model = LogisticRegression(solver='liblinear')\n",
    "model.fit(X_train, y_train)\n",
    "\n",
    "# evaluate model\n",
    "accuracy = clf.score(X_test, y_test)\n",
    "print(\"accuracy: %.4f\" % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The popular library scikit learn has many models included.  Training different types of models is a trivial code change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "model = DecisionTreeClassifier()\n",
    "model.fit(X_train, y_train)\n",
    "\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "model = KNeighborsClassifier()\n",
    "model.fit(X_train, y_train)\n",
    "\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "model = MLPClassifier(hidden_layer_sizes=[10])\n",
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Furthermore, each model has many different \"hyperparameters\" which affect the runtime and accuracy of the model.  (See for example <https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html>.)\n",
    "\n",
    "Since this is a DATA SCIENCE class, the focus will be on determining which model to use and which hyperparameters.\n",
    "We will not focus on how to implement these models.\n",
    "\n",
    "If this were a COMPUTER SCIENCE class, the focus would be more on implementation that application."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Takeaway\n",
    "\n",
    "1. The programming is EASY.\n",
    "1. Selecting the best algorithm and hyperparameters is HARD and requires a lot of math.\n",
    "\n",
    "   That math is the focus of this course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}