Wisesight_Sentiment_Analysis_ML_Modeling.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "Wisesight Sentiment Analysis - ML Modeling",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8AF08ssF6z5K"
      },
      "source": [
        "# Wisesight Sentiment Analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o-tHtMyc6z5L"
      },
      "source": [
        "This notebook details the steps taken to create a sentiment analyzer using data from [Wisesight Sentiment Corpus](https://github.com/PyThaiNLP/wisesight-sentiment/). Evaluation metric is overall accuracy across `neg`ative, `pos`itive, `neu`tral and `q`uestion classes. We give examples using logistic regression and [ULMFit](https://arxiv.org/abs/1801.06146).\n",
        "\n",
        "The results for logistic regression, [fastText](https://fasttext.cc/), ULMFit, ULMFit with semi-supervised data are as follows:\n",
        "\n",
        "| Model               | Public Accuracy | Private Accuracy |\n",
        "|---------------------|-----------------|------------------|\n",
        "| Logistic Regression | 0.72781         | 0.7499           |\n",
        "| fastText            | 0.63144         | 0.6131           |\n",
        "| ULMFit              | 0.71259         | 0.74194          |\n",
        "| ULMFit Semi-supervised    | 0.73119     | 0.75859      |\n",
        "| ULMFit Semi-supervised Repeated One Time    | **0.73372**     | **0.75968**      |\n",
        "\n",
        "* For more information about the Kaggle competition, which this notebook based upon, see [1st Place Solution](https://www.kaggle.com/c/wisesight-sentiment/discussion/83564).\n",
        "* For more updated version of data, including a tokenized `wisesight-1000` evaluation set, see [Wisesight Sentiment Corpus](https://github.com/PyThaiNLP/wisesight-sentiment).\n",
        "* Original -> https://github.com/PyThaiNLP/wisesight-sentiment/blob/master/kaggle-competition/competition.ipynb"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LlSXHPBBaF3i"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bNjkuQK46z5M",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "50b4b920-5d85-40ab-c889-9a27991e3617"
      },
      "source": [
        "#uncomment if you are running from google colab\n",
        "!pip install sklearn_crfsuite -q\n",
        "!pip install emoji -q\n",
        "!pip install https://github.com/PyThaiNLP/pythainlp/archive/dev.zip -q\n",
        "!wget https://github.com/PyThaiNLP/wisesight-sentiment/archive/master.zip\n",
        "!unzip master.zip\n",
        "!mkdir wisesight_data; ls\n",
        "!cd wisesight-sentiment-master/kaggle-competition; ls"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 747kB 5.1MB/s \n",
            "\u001b[K     |████████████████████████████████| 133kB 6.7MB/s \n",
            "\u001b[K     - 12.7MB 138kB/s\n",
            "\u001b[?25h  Building wheel for pythainlp (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "--2021-05-14 06:15:49--  https://github.com/PyThaiNLP/wisesight-sentiment/archive/master.zip\n",
            "Resolving github.com (github.com)... 140.82.114.3\n",
            "Connecting to github.com (github.com)|140.82.114.3|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://codeload.github.com/PyThaiNLP/wisesight-sentiment/zip/master [following]\n",
            "--2021-05-14 06:15:49--  https://codeload.github.com/PyThaiNLP/wisesight-sentiment/zip/master\n",
            "Resolving codeload.github.com (codeload.github.com)... 140.82.114.9\n",
            "Connecting to codeload.github.com (codeload.github.com)|140.82.114.9|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: unspecified [application/zip]\n",
            "Saving to: ‘master.zip’\n",
            "\n",
            "master.zip              [   <=>              ]   6.07M  4.80MB/s    in 1.3s    \n",
            "\n",
            "2021-05-14 06:15:50 (4.80 MB/s) - ‘master.zip’ saved [6365386]\n",
            "\n",
            "Archive:  master.zip\n",
            "61a0409a1b648121d25658044c1f37208080e170\n",
            "   creating: wisesight-sentiment-master/\n",
            "  inflating: wisesight-sentiment-master/.gitignore  \n",
            "  inflating: wisesight-sentiment-master/LICENSE  \n",
            "  inflating: wisesight-sentiment-master/README.md  \n",
            "  inflating: wisesight-sentiment-master/exploration.ipynb  \n",
            "   creating: wisesight-sentiment-master/huggingface/\n",
            " extracting: wisesight-sentiment-master/huggingface/data.zip  \n",
            "   creating: wisesight-sentiment-master/kaggle-competition/\n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/README.md  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/competition.ipynb  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/test.txt  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/test_label.txt  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/test_majority.csv  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/test_solution.csv  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/text_generation.ipynb  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/train.txt  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/train_label.txt  \n",
            "  inflating: wisesight-sentiment-master/kaggle-competition/train_model.py  \n",
            "  inflating: wisesight-sentiment-master/neg.txt  \n",
            "  inflating: wisesight-sentiment-master/neu.txt  \n",
            "  inflating: wisesight-sentiment-master/pos.txt  \n",
            "  inflating: wisesight-sentiment-master/process_huggingface.ipynb  \n",
            "  inflating: wisesight-sentiment-master/q.txt  \n",
            "   creating: wisesight-sentiment-master/word-tokenization/\n",
            "  inflating: wisesight-sentiment-master/word-tokenization/README.md  \n",
            "  inflating: wisesight-sentiment-master/word-tokenization/data-preparation-and-post-processing.ipynb  \n",
            "  inflating: wisesight-sentiment-master/word-tokenization/wisesight-1000-samples-tokenised.label  \n",
            "  inflating: wisesight-sentiment-master/word-tokenization/wisesight-1000-samples-tokenised.txt  \n",
            "  inflating: wisesight-sentiment-master/word-tokenization/wisesight-160-samples-tokenised.label  \n",
            "  inflating: wisesight-sentiment-master/word-tokenization/wisesight-160-samples-tokenised.txt  \n",
            "master.zip  sample_data  wisesight_data  wisesight-sentiment-master\n",
            "competition.ipynb  test_majority.csv  text_generation.ipynb  train.txt\n",
            "README.md\t   test_solution.csv  train_label.txt\n",
            "test_label.txt\t   test.txt\t      train_model.py\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dzk5UgF_OZFa"
      },
      "source": [
        "# snippet to install thai font in matplotlib from https://gist.github.com/korakot/9d7f5db632351dc92607fdec72a4953f\n",
        "# import matplotlib\n",
        "# !wget https://github.com/Phonbopit/sarabun-webfont/raw/master/fonts/thsarabunnew-webfont.ttf\n",
        "# !cp thsarabunnew-webfont.ttf /usr/local/lib/python3.6/dist-packages/matplotlib/mpl-data/fonts/ttf/\n",
        "# !cp thsarabunnew-webfont.ttf /usr/share/fonts/truetype/\n",
        "# matplotlib.font_manager._rebuild()\n",
        "# matplotlib.rc('font', family='TH Sarabun New')"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zs8wtP0m6z5O"
      },
      "source": [
        "%load_ext autoreload\n",
        "%autoreload 2\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "from pythainlp import word_tokenize\n",
        "from tqdm import tqdm_notebook\n",
        "# from pythainlp.ulmfit import process_thai\n",
        "\n",
        "#viz\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "import re\n",
        "import emoji\n",
        "\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\")"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "71ZMt4_h6z5U"
      },
      "source": [
        "## Process Text Files to CSVs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4NNjkF3i6z5U",
        "scrolled": true,
        "outputId": "6dff53fa-1a6d-49d5-e989-b2996cc3d6f1"
      },
      "source": [
        "with open(\"wisesight-sentiment-master/kaggle-competition/train.txt\") as f:\n",
        "    texts = [line.strip() for line in f.readlines()]\n",
        "\n",
        "with open(\"wisesight-sentiment-master/kaggle-competition/train_label.txt\") as f:\n",
        "    categories = [line.strip() for line in f.readlines()]\n",
        "\n",
        "all_df = pd.DataFrame({\"category\":categories, \"texts\":texts})\n",
        "all_df.to_csv('wisement-sentiment.csv',index=False)\n",
        "all_df.shape"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(24063, 2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E3Ww3CIi6z5X",
        "outputId": "764d2f84-f89c-4d8a-abbb-62978b2d4f09"
      },
      "source": [
        "with open(\"wisesight-sentiment-master/kaggle-competition/test.txt\") as f:\n",
        "    texts = [line.strip() for line in f.readlines()]\n",
        "\n",
        "test_df = pd.DataFrame({\"category\":\"test\", \"texts\":texts})\n",
        "test_df.shape"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2674, 2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hWlGFHC-aAVE"
      },
      "source": [
        "# Text Processor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BL0e1HSuaDYY"
      },
      "source": [
        "def replace_url(text):\n",
        "    URL_PATTERN = r\"\"\"(?i)\\b((?:https?:(?:/{1,3}|[a-z0-9%])|[a-z0-9.\\-]+[.](?:com|net|org|edu|gov|mil|aero|asia|biz|cat|coop|info|int|jobs|mobi|museum|name|post|pro|tel|travel|xxx|ac|ad|ae|af|ag|ai|al|am|an|ao|aq|ar|as|at|au|aw|ax|az|ba|bb|bd|be|bf|bg|bh|bi|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|cr|cs|cu|cv|cx|cy|cz|dd|de|dj|dk|dm|do|dz|ec|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gg|gh|gi|gl|gm|gn|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|im|in|io|iq|ir|is|it|je|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|me|mg|mh|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|mv|mw|mx|my|mz|na|nc|ne|nf|ng|ni|nl|no|np|nr|nu|nz|om|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|ps|pt|pw|py|qa|re|ro|rs|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|Ja|sk|sl|sm|sn|so|sr|ss|st|su|sv|sx|sy|sz|tc|td|tf|tg|th|tj|tk|tl|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|us|uy|uz|va|vc|ve|vg|vi|vn|vu|wf|ws|ye|yt|yu|za|zm|zw)/)(?:[^\\s()<>{}\\[\\]]+|\\([^\\s()]*?\\([^\\s()]+\\)[^\\s()]*?\\)|\\([^\\s]+?\\))+(?:\\([^\\s()]*?\\([^\\s()]+\\)[^\\s()]*?\\)|\\([^\\s]+?\\)|[^\\s`!()\\[\\]{};:'\".,<>?«»“”‘’])|(?:(?<!@)[a-z0-9]+(?:[.\\-][a-z0-9]+)*[.](?:com|net|org|edu|gov|mil|aero|asia|biz|cat|coop|info|int|jobs|mobi|museum|name|post|pro|tel|travel|xxx|ac|ad|ae|af|ag|ai|al|am|an|ao|aq|ar|as|at|au|aw|ax|az|ba|bb|bd|be|bf|bg|bh|bi|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cd|cf|cg|ch|ci|ck|cl|cm|cn|co|cr|cs|cu|cv|cx|cy|cz|dd|de|dj|dk|dm|do|dz|ec|ee|eg|eh|er|es|et|eu|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gg|gh|gi|gl|gm|gn|gp|gq|gr|gs|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|im|in|io|iq|ir|is|it|je|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|me|mg|mh|mk|ml|mm|mn|mo|mp|mq|mr|ms|mt|mu|mv|mw|mx|my|mz|na|nc|ne|nf|ng|ni|nl|no|np|nr|nu|nz|om|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|ps|pt|pw|py|qa|re|ro|rs|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|Ja|sk|sl|sm|sn|so|sr|ss|st|su|sv|sx|sy|sz|tc|td|tf|tg|th|tj|tk|tl|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|us|uy|uz|va|vc|ve|vg|vi|vn|vu|wf|ws|ye|yt|yu|za|zm|zw)\\b/?(?!@)))\"\"\"\n",
        "    return re.sub(URL_PATTERN, 'xxurl', text)\n",
        "\n",
        "def replace_rep(text):\n",
        "    def _replace_rep(m):\n",
        "        c,cc = m.groups()\n",
        "        return f'{c}xxrep'\n",
        "    re_rep = re.compile(r'(\\S)(\\1{2,})')\n",
        "    return re_rep.sub(_replace_rep, text)\n",
        "\n",
        "def ungroup_emoji(toks):\n",
        "    res = []\n",
        "    for tok in toks:\n",
        "        if emoji.emoji_count(tok) == len(tok):\n",
        "            for char in tok:\n",
        "                res.append(char)\n",
        "        else:\n",
        "            res.append(tok)\n",
        "    return res\n",
        "\n",
        "def process_text(text):\n",
        "    #pre rules\n",
        "    res = text.lower().strip()\n",
        "    res = replace_url(res)\n",
        "    res = replace_rep(res)\n",
        "    \n",
        "    #tokenize\n",
        "    res = [word for word in word_tokenize(res) if word and not re.search(pattern=r\"\\s+\", string=word)]\n",
        "    \n",
        "    #post rules\n",
        "    res = ungroup_emoji(res)\n",
        "    \n",
        "    return res"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E5rbH6aJ6z5a"
      },
      "source": [
        "## Load Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6rWxBa-Ajn0w",
        "outputId": "954d1c8a-7bb6-460c-b524-acc36193c8df"
      },
      "source": [
        "all_df.texts"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0        ประเทศเราผลิตและส่งออกยาสูบเยอะสุดในโลกจิงป่าวคับ\n",
              "1                                                       คะ\n",
              "2                              อิเหี้ยออมทำกูอยากกินเอ็มเค\n",
              "3                                                      😅😅😅\n",
              "4                                  สวัสดีวันพุธ แนน อะไรนะ\n",
              "                               ...                        \n",
              "24058                                       แม่งควายล้วนนน\n",
              "24059                     ดอยสุเทพน้องง ไปหมดแล้วววว #pm25\n",
              "24060                             ค่าชุดอาจจะแพงกว่าส่วนลด\n",
              "24061                                รัฐต้องการแค่ภาษีครับ\n",
              "24062    ™ ได้พัฒนาระบบเข้าสู่รุ่นที่สองแล้ว มาดูกันสิว...\n",
              "Name: texts, Length: 24063, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yhQ6MX7U6z5a"
      },
      "source": [
        "all_df[\"processed\"] = all_df.texts.map(lambda x: \"|\".join(process_text(x)))\n",
        "all_df[\"wc\"] = all_df.processed.map(lambda x: len(x.split(\"|\")))\n",
        "all_df[\"uwc\"] = all_df.processed.map(lambda x: len(set(x.split(\"|\"))))\n",
        "\n",
        "test_df[\"processed\"] = test_df.texts.map(lambda x: \"|\".join(process_text(x)))\n",
        "test_df[\"wc\"] = test_df.processed.map(lambda x: len(x.split(\"|\")))\n",
        "test_df[\"uwc\"] = test_df.processed.map(lambda x: len(set(x.split(\"|\"))))"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iEDdQ9hH6z5c",
        "outputId": "ad0112e9-e756-4367-fd66-f7e2ab374a91"
      },
      "source": [
        "#prevalence\n",
        "all_df.category.value_counts() / all_df.shape[0]"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "neu    0.544612\n",
              "neg    0.255164\n",
              "pos    0.178698\n",
              "q      0.021527\n",
              "Name: category, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-jPw7Rbu6z5f"
      },
      "source": [
        "## Train-validation Split"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1LVJs5X1mRpk"
      },
      "source": [
        "#when finding hyperparameters\n",
        "from sklearn.model_selection import train_test_split\n",
        "train_df, valid_df = train_test_split(all_df, test_size=0.15, random_state=1412)\n",
        "train_df = train_df.reset_index(drop=True)\n",
        "valid_df = valid_df.reset_index(drop=True)"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GojEjj2k6z5m",
        "outputId": "95c3ad9c-eb7a-49fd-8147-9f7c309faeb2"
      },
      "source": [
        "#prevalence\n",
        "print(train_df[\"category\"].value_counts() / train_df.shape[0])"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "neu    0.544957\n",
            "neg    0.253557\n",
            "pos    0.180071\n",
            "q      0.021415\n",
            "Name: category, dtype: float64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sH1t3bal6z5o",
        "outputId": "bee9925c-618a-4e7b-bfd4-159e5759fc80"
      },
      "source": [
        "#prevalence\n",
        "print(valid_df[\"category\"].value_counts() / valid_df.shape[0])"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "neu    0.542659\n",
            "neg    0.264266\n",
            "pos    0.170914\n",
            "q      0.022161\n",
            "Name: category, dtype: float64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5eI-DEzW6z5r"
      },
      "source": [
        "#dependent variables\n",
        "y_train = train_df[\"category\"]\n",
        "y_valid = valid_df[\"category\"]"
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UOSQc4bTeutU"
      },
      "source": [
        "# Feature Extraction"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WctnHK1ObuYc"
      },
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.model_selection import RepeatedStratifiedKFold\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "import joblib\n",
        "\n",
        "tfidf = TfidfVectorizer(tokenizer=word_tokenize, ngram_range=(1,2), min_df=20, sublinear_tf=True)\n",
        "\n",
        "tfidf_fit = tfidf.fit(all_df[\"texts\"])\n",
        "\n",
        "tf = joblib.dump(tfidf_fit, \"tfidf.pickle\")\n",
        "\n",
        "text_train = tfidf_fit.transform(train_df[\"texts\"])\n",
        "text_valid = tfidf_fit.transform(valid_df[\"texts\"])\n",
        "text_test = tfidf_fit.transform(test_df[\"texts\"])\n",
        "\n",
        "tfidf_fit_t = tfidf_fit.transform(all_df[\"texts\"])\n",
        "\n",
        "\n",
        "scaler = StandardScaler()\n",
        "\n",
        "scaler_fit = scaler.fit(all_df[[\"wc\",\"uwc\"]].astype(float))\n",
        "\n",
        "sc = joblib.dump(scaler_fit, \"scaler.pickle\")\n",
        "\n",
        "num_train = scaler_fit.transform(train_df[[\"wc\",\"uwc\"]].astype(float))\n",
        "num_valid = scaler_fit.transform(valid_df[[\"wc\",\"uwc\"]].astype(float))\n",
        "num_test = scaler_fit.transform(test_df[[\"wc\",\"uwc\"]].astype(float))\n",
        "\n",
        "scaler_fit_t = scaler_fit.transform(all_df[[\"wc\",\"uwc\"]].astype(float))"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YV8P7oCVdNL5",
        "outputId": "8a075653-3f0c-4a51-87bf-6be67ad828c1"
      },
      "source": [
        "#concatenate text and word count features\n",
        "#split dataset\n",
        "X_train = np.concatenate([num_train, text_train.toarray()],axis=1)\n",
        "X_valid = np.concatenate([num_valid, text_valid.toarray()],axis=1)\n",
        "X_test = np.concatenate([num_test, text_test.toarray()],axis=1)\n",
        "X_train.shape, X_valid.shape"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((20453, 6159), (3610, 6159))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 424
        },
        "id": "-vYfzYjJbkHf",
        "outputId": "17799f01-ce48-4309-e4d5-3db52cf53e64"
      },
      "source": [
        "#whole dataset\n",
        "tfidf_fit_df = pd.DataFrame(tfidf_fit_t.toarray())\n",
        "scaler_fit_df = pd.DataFrame(scaler_fit_t)\n",
        "target = all_df['category']\n",
        "df = pd.concat([scaler_fit_df, tfidf_fit_df], axis=1)\n",
        "df"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>10</th>\n",
              "      <th>11</th>\n",
              "      <th>12</th>\n",
              "      <th>13</th>\n",
              "      <th>14</th>\n",
              "      <th>15</th>\n",
              "      <th>16</th>\n",
              "      <th>17</th>\n",
              "      <th>18</th>\n",
              "      <th>19</th>\n",
              "      <th>20</th>\n",
              "      <th>21</th>\n",
              "      <th>22</th>\n",
              "      <th>23</th>\n",
              "      <th>24</th>\n",
              "      <th>25</th>\n",
              "      <th>26</th>\n",
              "      <th>27</th>\n",
              "      <th>28</th>\n",
              "      <th>29</th>\n",
              "      <th>30</th>\n",
              "      <th>31</th>\n",
              "      <th>32</th>\n",
              "      <th>33</th>\n",
              "      <th>34</th>\n",
              "      <th>35</th>\n",
              "      <th>36</th>\n",
              "      <th>37</th>\n",
              "      <th>...</th>\n",
              "      <th>6117</th>\n",
              "      <th>6118</th>\n",
              "      <th>6119</th>\n",
              "      <th>6120</th>\n",
              "      <th>6121</th>\n",
              "      <th>6122</th>\n",
              "      <th>6123</th>\n",
              "      <th>6124</th>\n",
              "      <th>6125</th>\n",
              "      <th>6126</th>\n",
              "      <th>6127</th>\n",
              "      <th>6128</th>\n",
              "      <th>6129</th>\n",
              "      <th>6130</th>\n",
              "      <th>6131</th>\n",
              "      <th>6132</th>\n",
              "      <th>6133</th>\n",
              "      <th>6134</th>\n",
              "      <th>6135</th>\n",
              "      <th>6136</th>\n",
              "      <th>6137</th>\n",
              "      <th>6138</th>\n",
              "      <th>6139</th>\n",
              "      <th>6140</th>\n",
              "      <th>6141</th>\n",
              "      <th>6142</th>\n",
              "      <th>6143</th>\n",
              "      <th>6144</th>\n",
              "      <th>6145</th>\n",
              "      <th>6146</th>\n",
              "      <th>6147</th>\n",
              "      <th>6148</th>\n",
              "      <th>6149</th>\n",
              "      <th>6150</th>\n",
              "      <th>6151</th>\n",
              "      <th>6152</th>\n",
              "      <th>6153</th>\n",
              "      <th>6154</th>\n",
              "      <th>6155</th>\n",
              "      <th>6156</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-0.263568</td>\n",
              "      <td>-0.227114</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>-0.625733</td>\n",
              "      <td>-0.769135</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-0.384290</td>\n",
              "      <td>-0.407788</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>-0.595553</td>\n",
              "      <td>-0.723966</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>-0.474831</td>\n",
              "      <td>-0.543293</td>\n",
              "      <td>0.148379</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24058</th>\n",
              "      <td>-0.505011</td>\n",
              "      <td>-0.588461</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24059</th>\n",
              "      <td>-0.323929</td>\n",
              "      <td>-0.317451</td>\n",
              "      <td>0.100392</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.140679</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24060</th>\n",
              "      <td>-0.474831</td>\n",
              "      <td>-0.543293</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24061</th>\n",
              "      <td>-0.505011</td>\n",
              "      <td>-0.588461</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24062</th>\n",
              "      <td>0.008056</td>\n",
              "      <td>0.134232</td>\n",
              "      <td>0.127761</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>24063 rows × 6159 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "           0         1         0     1     2     ...  6152  6153  6154  6155  6156\n",
              "0     -0.263568 -0.227114  0.000000   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "1     -0.625733 -0.769135  0.000000   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "2     -0.384290 -0.407788  0.000000   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "3     -0.595553 -0.723966  0.000000   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "4     -0.474831 -0.543293  0.148379   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "...         ...       ...       ...   ...   ...  ...   ...   ...   ...   ...   ...\n",
              "24058 -0.505011 -0.588461  0.000000   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "24059 -0.323929 -0.317451  0.100392   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "24060 -0.474831 -0.543293  0.000000   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "24061 -0.505011 -0.588461  0.000000   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "24062  0.008056  0.134232  0.127761   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "\n",
              "[24063 rows x 6159 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z0FIaYOhe0C8"
      },
      "source": [
        "# SMOTE\n",
        "https://machinelearningmastery.com/smote-oversampling-for-imbalanced-classification/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ndyjlnm2hk28"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dcF9iTPbbkHr"
      },
      "source": [
        "from sklearn.preprocessing import LabelEncoder\n",
        "data = df.values\n",
        "# split into input and output elements\n",
        "X, y =  data, target\n",
        "# label encode the target variable \n",
        "# neg 0, neu 1, pos 2, q 3\n",
        "y = LabelEncoder().fit_transform(y)"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pxOeivbabkHr"
      },
      "source": [
        "# transform the dataset\n",
        "from imblearn.over_sampling import SMOTE\n",
        "oversample = SMOTE()\n",
        "X, y = oversample.fit_resample(X, y)"
      ],
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LilMC1lbbkHr"
      },
      "source": [
        "from sklearn.model_selection import cross_val_score\n",
        "from numpy import std\n",
        "from numpy import mean\n",
        "# evaluate a model\n",
        "def evaluate_model(X, y, model):\n",
        "\t# define evaluation procedure\n",
        "\tcv = RepeatedStratifiedKFold(n_splits=5, n_repeats=3, random_state=1)\n",
        "\t# evaluate model\n",
        "\tscores = cross_val_score(model, X, y, scoring='accuracy', cv=cv, n_jobs=-1)\n",
        "\treturn scores"
      ],
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iy4leuNHdkuI"
      },
      "source": [
        "# Logistic Regression"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZS3DTG02bkHs",
        "outputId": "651e2f5e-47fd-4d14-d117-974d1a5a5006"
      },
      "source": [
        "# Logitic Regression\n",
        "import time\n",
        "lgr = LogisticRegression(C=2., penalty=\"l2\", solver=\"liblinear\", dual=False, multi_class=\"ovr\")\n",
        "start = time.time()\n",
        "scores = evaluate_model(X, y, lgr)\n",
        "end = time.time() - start\n",
        "print(\"Model Evaluation Time :\" , end)\n",
        "# summarize performance\n",
        "print('Mean Accuracy: %.3f (%.3f)' % (mean(scores), std(scores)))\n",
        "print(scores)"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model Evaluation Time : 32.20143723487854\n",
            "Mean Accuracy: 0.800 (0.004)\n",
            "[0.79826402 0.80017169 0.80055322 0.79463945 0.80455933 0.79969477\n",
            " 0.7983594  0.80227013 0.79483022 0.80780237 0.80074399 0.80560855\n",
            " 0.79912247 0.80179321 0.79225486]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Va2f-HuuU9vQ",
        "outputId": "b323b592-744a-4054-ef03-fdd7bf107892"
      },
      "source": [
        "lgr.fit(X, y)"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=2.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
              "                   multi_class='ovr', n_jobs=None, penalty='l2',\n",
              "                   random_state=None, solver='liblinear', tol=0.0001, verbose=0,\n",
              "                   warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fZKQJuS_fLSJ"
      },
      "source": [
        "# Save ML Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eCl8DpRpfKRQ",
        "outputId": "f84745a5-8f8d-4038-82da-5071632d65d7"
      },
      "source": [
        "from sklearn.externals import joblib\n",
        "# Save model to file in the current working directory\n",
        "joblib_file = \"joblib_lgr.pkl\"  \n",
        "joblib.dump(lgr, joblib_file)"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['joblib_lgr.pkl']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "usd-VPRtfUYO"
      },
      "source": [
        "# Model Evaluation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gOKbzjpGfYbJ"
      },
      "source": [
        "from sklearn.metrics import classification_report"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jXvPv2XvzsJp"
      },
      "source": [
        "y_pred = lgr.predict(X_valid)"
      ],
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BEcwK9Xtj1VI"
      },
      "source": [
        "y_valid_b = LabelEncoder().fit_transform(y_valid)"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XIkfF8HIvcL-",
        "outputId": "08f253f4-e843-4991-91f7-f4c18ded7f63"
      },
      "source": [
        "print(lgr.score(X_valid, y_valid_b))\n",
        "print(classification_report(y_pred, y_valid_b))"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.7626038781163434\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.85      0.76      0.80      1072\n",
            "           1       0.71      0.88      0.79      1579\n",
            "           2       0.76      0.61      0.67       769\n",
            "           3       1.00      0.42      0.59       190\n",
            "\n",
            "    accuracy                           0.76      3610\n",
            "   macro avg       0.83      0.67      0.71      3610\n",
            "weighted avg       0.78      0.76      0.76      3610\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "id": "lZID9Y2KZx2C",
        "outputId": "b191b2b2-5d2b-4cc0-9cbd-0df22df2da30"
      },
      "source": [
        "from sklearn.metrics import confusion_matrix\n",
        "# neg 0, neu 1, pos 2, q 3\n",
        "conf_mat = confusion_matrix(y_valid_b, y_pred)\n",
        "sns.heatmap(conf_mat, annot=True, fmt='d',\n",
        "            xticklabels=lgr.classes_, yticklabels=lgr.classes_)\n",
        "plt.ylabel('Actual')\n",
        "plt.xlabel('Predicted')\n",
        "plt.show()"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAEGCAYAAACaSwWnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3wVVfrH8c+TEEjoJXRQULAgu4hUG70XwZ9YFkVW2UXXjmvHXVZR11274ioIKNgAsQAWBAHBBoKCSDMGEAiEotJrcnN+f9wJRAxwSXIzl8n37WtemXtm7p3njvDk8MyZM+acQ0REgiHO7wBERKTgKKmLiASIkrqISIAoqYuIBIiSuohIgBTzO4AjSWvRTsNyPC1SNvkdQszYtn+33yHEjIxQpt8hxIwD+9Msv5+R8fOqiHNOQvIp+T5etKinLiISIDHbUxcRKVRZIb8jKBBK6iIiAAEpZympi4gAzmX5HUKBUFIXEQHIUlIXEQkO9dRFRAJEF0pFRAJEPXURkeBwGv0iIhIgulAqIhIgKr+IiASILpSKiASIeuoiIgGiC6UiIgGiC6UiIsHhnGrqIiLBoZq6iEiAqPwiIhIg6qmLiARIKMPvCAqEkrqICKj8IiISKAEpv8T5HYDfSl/Rh6pvjqbqG6OoOPR+KJ5AqT69qTbxVWrNm0lcubK/2b/c7TdRbeKrVHntJRJOr+9T1NE34LqrmPHle8z8chJ/ub7fb7Zdd2N/1m9dSoWK5X2KrvCUKFGC2XPeY+7cj5i/YBqD7x8EwLTpE/hq7od8NfdDUlfOY9z4ET5HGn0jhj9O2rpFLPz2k4Nt/xpyB98smM78rz/mgw9ep3r1qj5GmE9ZWZEvMaxIJ/W4ysmUvvxiNv35ejb1HQBxcZTs2I4Di5ew5eY7yNyw8Tf7J57XgoTaNdnYpx/bHn2SCnfd5lPk0XX6mfXo278P3dtfQccL/48OnVtTp+5JANSoWY1Wbc8nbd0Gn6MsHPv376db1760bNmVc1t2o2PH1jRr1phOHS/j3JbdOLdlN+bN+5bJk6b6HWrUjX31LXr0vOo3bU88+SJNmnakWfPOfPjhDAYPPoH/TiipB0R8PFaiBMTHYYklCP38CxkpqYTSN/1u18RW57H7o+kAHFiyHCtTmrhKFQs74qirf9opLFywmH179xEKhZj7xQK69uwAwL8evpuH//UEzjmfoyw8u3fvASAhoRgJCcVwHPruZcqUpnXr85gyZZpf4RWazz+fx9at237TtnPnroPrpUomndB/LlwoI+IllkWtpm5mZwC9gJpe03pgsnNuebSOebyytvzMrtcnUH3SONz+/eyft4D98xYccf/4ysmENm0++Dq0eQvxlZPJ+uXXwgi30KxYnsrd999KhQrl2LtvP+06Xsh3i5bSqWtb0tM3sWzJD36HWKji4uL44sv3OeWUkxkx/FUWzF90cFvPnp349NMvfpPcipoHH7iLK6/sw44dO+jY6TK/w8k71dSPzMzuBsYBBnztLQa8aWb3ROOYeWFlSpPY6nw2XtyX9O6XYkmJlOzSwe+wfJeasornnxnFG++8xOsTh7N0yQqKFy/OzbcP5PF/D/M7vEKXlZXFuS27cVr9c2nStBENGpx2cNull13EWxMm+xid//455L+cWq85b775Ljf87Rq/w8k7lV+OagDQzDn3qHPuNW95FGjubcuVmQ00swVmtuD1zdGv2SY2a0JoQzpZ27ZDKMTeWZ9R/A9nHXH/0Jafia9a5eDr+CqVCW35Oepx+mHca+/Qte1lXNK9P9u37SBlRSonnVyT6Z+9w9zvplG9RlU+nj2RylWS/Q610GzfvoM5c76iY8fWAFSqVIEmTRoxdeosnyOLDW+Oe5eLL+7qdxh557IiX47BzEab2WYzW5Kj7TEzW2Fmi83sXTMrn2PbvWaWamY/mFnnHO1dvLbUSDvE0UrqWUCNXNqre9ty5Zwb4Zxr6pxremWV3N5esEKbNlG8YYNwTR0o0ewcMn5ae8T99332JaW6dgSgeMMzcbt2B670kq1ScvhaQY1a1enaowNvvTmJRqe1omWjTrRs1In0DZvo3LoPWzYH85datuTkipTzRkAlJpagXbsL+CFlJQC9L+7G1I9msn//fj9D9FW9enUPrvfs2ZkffljpYzT5VLA99VeALoe1TQcaOuf+CKQA9wKYWQPgCuAs7z3/M7N4M4sHnge6Ag2AP3n7HlW0auq3ATPM7Edgndd2ElAPuClKxzxuB5auYO/M2VQZOxxCIQ6kpLL7vfcpfdnFlO53BfEVK1L19ZHs+3IeWx95gn1fzCPxvBZUe/s13L59/Dr0v35/hah5aezTVKhQnszMTAbf+RA7duz0OyRfVKtWhREvPUF8XBxxcXG8/c4HTP1oJgB9+vTkySde8DnCwvPq2GG0anUuyckVWbVyPg8OfYKuXdpx2mmnkJXlWLs2jRtvutfvMPOuAGvqzrk5ZlbnsLacV9PnAn289V7AOOfcfmC1maUSrmoApDrnVgGY2Thv32VHO7ZF62q1mcV5geW8UDrfRTi/ZVqLdifuZfQC1iLl9yNxiqpt+3f7HULMyAjIQx0KwoH9aZbfz9j7wdMR55ySPQZdBwzM0TTCOfebmxW8pP6+c67h4e83synAeOfca2Y2DJjrnHvN2zYK+MjbtYtz7i9eez+ghXPuqB3jqI1+cc5lEf5tJCIS+46jp+4l8DzdcWZmg4FM4PW8vP9YNE2AiAgUyqgWM/sz0ANo7w6VSdYDtXPsVstr4yjtR6Sbj0REoEBHv+TGzLoAdwEXOef25Ng0GbjCzEqYWV2gPuFh4POB+mZW18yKE76Yeszxs+qpi4hAgfbUzexNoA2QbGZpwBDCo11KANPNDMJ19Oudc0vNbALhC6CZwI3Z1x7N7CbgYyAeGO2cW3qsYyupi4hAQY9++VMuzaOOsv/DwMO5tH8IfHg8x1ZSFxEByAzGaCIldRERgBN4MrKclNRFRCDm53SJlJK6iAgoqYuIBEpApt5VUhcRAQhFNINJzFNSFxEBlV9ERAJFSV1EJEBUUxcRCQ6XpXHqIiLBofKLiEiAaPSLiEiAqKcuIhIgSuoiIgGiCb1ERAJEPXURkQDRkMboujQtGCe4IPz04xS/Q4gZFzW+0e8QYsainWv8DiFYNPpFRCQ4nMovIiIBovKLiEiAaO4XEZEACUhPPc7vAEREYkJmKPLlGMxstJltNrMlOdoqmtl0M/vR+1nBazcze9bMUs1ssZmdk+M9/b39fzSz/pF8DSV1EREIl18iXY7tFaDLYW33ADOcc/WBGd5rgK5AfW8ZCLwA4V8CwBCgBdAcGJL9i+BolNRFRCBcfol0OQbn3Bzg18OaewFjvPUxQO8c7WNd2FygvJlVBzoD051zvzrntgLT+f0vit9RTV1EhOMb0mhmAwn3qrONcM6NOMbbqjrn0r31jUBVb70msC7Hfmle25Haj0pJXUQEjutCqZfAj5XEj/Z+Z2ZRuTKr8ouICBRo+eUINnllFbyfm7329UDtHPvV8tqO1H5USuoiIhCeJiDSJW8mA9kjWPoDk3K0X+2NgmkJbPfKNB8DncysgneBtJPXdlQqv4iIULDPKDWzN4E2QLKZpREexfIoMMHMBgBrgMu83T8EugGpwB7gGgDn3K9mNhSY7+33oHPu8Iuvv6OkLiICBXrzkXPuT0fY1D6XfR2Q60x1zrnRwOjjObaSuogIaD51EZFACcg0AUrqIiKgpC4iEiQupPKLiEhwqKcuIhIcBTmk0U9K6iIioJ66iEigBKOkrqQuIgLgMoOR1ZXURURAPfUgqFKjMv985l4qJlfAOZj0+vtMGPU27Xq0ZsDtf6ZO/ZMY0P1vrFiccvA9V9/Ul55XdCOUFeKpfwxj3uz5RzlCbLv/kSeZ88XXVKxQnvdeexGA50aMZebnXxFncVSsUI6HB/+dKpUrsX3HTv7x76dYtz6dEsWLM/S+QdQ/pQ4AO3buYsijT5O6ag2YMfS+QZzd8Ewfv1n+JFdP5o6n76BCcgWcc3z0xkdMGh2ee+miP19Ej/49yApl8fXMrxn9yGiq1KrCiFkjSFuZBsCKb1cw7L5hfn6FqBl4w9X07dcH5xzLl6Uw6MbBNGtxDv8cegcWF8eeXbu59YbB/LR6rd+hHjddKA2AUGaIZx94gZQlP1KyVBIvTx3O13MWsHLFau796z+5+9Hbf7N/nfon06FXO/q2u4bkqpV4dtzjXH7h1WSdoLcX9+7Wkb6XXMR9Qx8/2HbNlZdw88CrAXjtrUm88PIbDLnrZl4aO54z6p/Ks//+J6vWrOPhJ55n1LOPAvDo0y9yfoumPPXw/WRkZLB3335fvk9BCYVCvDT0JVYuWUlSqSSe/fBZFn62kPLJ5WnZqSU3dr6RjAMZlKtU7uB70tekc1OXm3yMOvqqVa/CgOuuonWLnuzbt5/hLz9Jr0u6ccvtA7mm7038mLKK/gOu4LY7r+O2Gwb7He7xOzH/Gv9OkZ5695fNv5Ky5EcA9uzey08/rqVytWTWpK5l7cp1v9u/Vefz+WTSTDIOZJC+biNpP22gQeMzCjvsAtP07D9QrmyZ37SVLlXq4PrevfswC6+v/GktLc5pBMApJ9dmffomfv51Kzt37eab75ZwSc/OACQkJFC2TOnC+QJRsnXzVlYuWQnA3t17WZe6jkrVKtG9X3cm/G8CGQcyANj+y3Y/w/RFfHw8iYmJxMfHk5SUyKb0zTjnKO39Py9btgyb0rf4HGXeuCwX8RLLCr2nbmbXOOdeLuzjHku1WlU5rWE9li5cfsR9KldLZsm3yw6+3pK+hcrVkgsjvEL1zPBXmDx1BmVKlWL0c+He+On1TuGT2V/Q5OyGfL/sB9I3bWbT5p+Jj4ujQvly3P/wk/yQuooGp9fnntuup2RSos/fomBUqVWFU886lR8W/sCAwQNo2Lwh/e/qT8b+DEY+NJKU78KluWq1qzHso2Hs2bWHMY+NYenXS32OvOBtTN/Mi8NeZsGSGezbt4/ZM79k9qwvueOWf/LaWy+yb+8+du3cTfeOV/gdat6op55nDxxpg5kNNLMFZrZg0+4NhRZQUslE/v3Sgzw95Hn27NpTaMeNVbde92dmvPsq3Tu15Y23pwDwl36XsnPXbi7pfyOvT5zMGfVPJT4ujsxQiOUpqVx+cXcmvvI8SUmJjHp1gs/foGAklkzk/uH3M/xfw9mzaw/xxeIpU74Mgy4axMiHR3Lv/+4Fwj37q1tczU1db2LEgyO4+7m7KVm6pM/RF7xy5crSuVs7WjTqyNlntKFkqSQuuawnA2+4mqsuvZ4mZ7Vj3Ovv8q+H7/Y71DxxmZEvsSwqSd3MFh9h+Z5DD1v9HefcCOdcU+dc06qlakQjtN+JLxbPIy89yMfvfsLsjz476r5bNv5M1RpVDr6uXL0yWzb+HO0QfdOjU1s++fQLIFyWeWjw7bw95nn+/Y872LptO7VqVqNalWSqVk7mj2eFy1Cd2lzAspRUP8MuEPHF4rl/xP3Mem8WX079EoCf03/mi4/C5yNlUQrOOcpVLEfGgQx2btsJQOr3qaSvSafmKcd8PvAJ58I257J2zXp++WUrmZmZfDhlOs1aNKZBw9NZ+M1iACa/+xHNmjf2OdK8cVmRL7EsWj31qsDVQM9cll+idMw8GfzEXaxJXcO4EW8dc9/Ppn1Jh17tSCieQPXa1ahdtybLFq4ohCgLz5p1hx6BOPOzr6h7ci0gPMIlIyNcS357ylSanP0HSpcqRXKlilSrUpnVa8IjP+Z+s4hT65xU+IEXsNseu411P67j3ZfePdj21cdf0ei88HWFmnVrUiyhGNt/3U65iuWIiwv/Vap2UjVq1K1B+tr0XD/3RLY+LZ0mTRuR5JXWLmjdkpQfVlK2bBlOOfVkAFq1PZcfU1b6GWbeZR3HEsOiVVN/HyjtnFt0+AYz+zRKxzxuf2zWkK59OpG6bCVjpr0EwIuPjqR48QRuf+gWylcsxxNj/03K0pUMuvIuVqf8xIwps3hj1suEQiEeH/zMCTvyBeDOIY8yf+Fitm3bQfveV3HDgH589tV8flqbhsUZNapV4Z933gzAqjXrGPzQExhwat2TefDe2w5+zn2D/sbdD/yXjMwMateoztD7Bvn0jQrGWc3OokOfDqxevpphU8NDE8f8ZwzTxk9j0OODeOGTF8g8kMkTg54AoGGLhvT7ez8yMzNxWY5h9w5j17Zdfn6FqFj4zWLenzyNabMnkpkZYsn3y3ntlQmkb9jIyLHPkOWy2L5tB4NuvN/vUPMk1nvgkbLwk5Riz7k128ZmYD6Y890ov0OIGRc1zvWpX0XSop1r/A4hZqRvW2b5/YzN7VtHnHOqzJid7+NFS5Eepy4iks2FYjZPHxcldRERglN+UVIXEQFcVjB66kX6jlIRkWwFOaTRzAaZ2VIzW2Jmb5pZopnVNbN5ZpZqZuPNrLi3bwnvdaq3vU5+voeSuogI4JxFvByNmdUEbgGaOucaAvHAFcB/gKecc/WArcAA7y0DgK1e+1PefnmmpC4iQoHffFQMSDKzYkBJIB1oB0z0to8BenvrvbzXeNvbm1mea0FK6iIiQFbIIl5yTmniLQOzP8c5tx54HFhLOJlvB74Btjl3cJKBNCD7tuOawDrvvZne/pXy+j10oVREhOO7UOqcGwGMyG2bmVUg3PuuC2wD3gK6FECIEVFSFxGhQEe/dABWO+e2AJjZO8D5QHkzK+b1xmsB2XNyrAdqA2leuaYc+ZhOReUXERHAuciXY1gLtDSzkl5tvD2wDJgF9PH26Q9M8tYne6/xts90+bjV/4g9dTN7DjjiBzvnbsnrQUVEYk1B9dSdc/PMbCLwLZAJLCRcqvkAGGdmD3lt2fN/jAJeNbNU4FfCI2Xy7GjllwX5+WARkRPJsYYqHt9nuSHAkMOaVwHNc9l3H3BpQR37iEndOTfmSNtERIImVFTmfjGzysDdQAPg4DPKnHPtohiXiEihKsieup8iuVD6OrCc8PCcB4CfgPlRjElEpNC5LIt4iWWRJPVKzrlRQIZzbrZz7lrCd0aJiARGAY5+8VUk49QzvJ/pZtYd2ABUjF5IIiKFL9Z74JGKJKk/ZGblgL8DzwFlgRP7eWUiIocJZQXjtp1jJnXn3Pve6nagbXTDERHxR6yXVSIVyeiXl8nlJiSvti4iEghZARn9Ekn55f0c64nAxYTr6iIigRGUIY2RlF/ezvnazN4EPo9aRCIiPigy5Zdc1AeqFHQgh1u2fW20D3HCOKleD79DiBk3lWvsdwgx45M9i/0OIVCKTPnFzHby25r6RsJ3mIqIBEZRGv1SpjACERHxU0CqL8e+o9TMZkTSJiJyIstyFvESy442n3oi4QemJnuPZ8r+JmU59Gw9EZFAKAqjX64DbgNqEH5oavY33gEMi3JcIiKFKsvvAArI0eZTfwZ4xsxuds49V4gxiYgUOkcweuqRXO7NMrPy2S/MrIKZ3RDFmERECl2ms4iXWBZJUv+rc25b9gvn3Fbgr9ELSUSk8Dks4iWWRXLzUbyZWfbTrc0sHige3bBERApX4GvqOUwFxpvZcO/1dcBH0QtJRKTwxXoPPFKRJPW7gYHA9d7rxUC1qEUkIuKDoPTUj1lTd85lAfMIP5u0OeFH2S2PblgiIoUrhEW8HIuZlTeziWa2wsyWm9m5ZlbRzKab2Y/ezwrevmZmz5pZqpktNrNz8vM9jpjUzew0MxtiZisIP/FoLYBzrq1zTuPURSRQsizyJQLPAFOdc2cAjQh3hO8BZjjn6gMzvNcAXQlPlFifcFXkhfx8j6P11FcQ7pX3cM5d4I1VD+XnYCIisSoLi3g5Gu/xn62AUQDOuQPeCMJewBhvtzFAb2+9FzDWhc0FyptZ9bx+j6Ml9f8D0oFZZvaSmbWHgFxJEBE5jDuOxcwGmtmCHMvAHB9VF9gCvGxmC81spJmVAqo659K9fTYCVb31msC6HO9PIx9TsRztjtL3gPe8YHoRnjKgipm9ALzrnJuW14OKiMSa47lQ6pwbAYw4wuZiwDnAzc65eWb2DIdKLdnvd2YWlYkhI7lQuts594ZzridQC1iI5lMXkYDJMot4OYY0IM05N897PZFwkt+UXVbxfm72tq8Haud4fy2vLU+Oa1Z459xW59wI51z7vB5QRCQWhY5jORrn3EZgnZmd7jW1B5YBk4H+Xlt/YJK3Phm42hsF0xLYnqNMc9zy8jg7EZHAiXBUS6RuBl43s+LAKuAawp3oCWY2AFgDXObt+yHQDUgF9nj75pmSuogIHHNUy/Fwzi0Cmuay6XdVDm8KlhsL6thK6iIiBOdxdkrqIiIUePnFN0rqOSxeOptdu3YTCoUIZYZo06o3Qx+6hy7d2nHgQAarV6/lxuvvYvv2nX6HGnUDb7iavv364Jxj+bIUBt04mKYtGjNk6J0kJCSw+Lul3H7TPwiFgns/msUZ177/EDs3bmXCtY8D0ObOSzmjWwtcVhbfvDqDBa98TMvrutOw1/nh9xSLI7leTZ5qfD37tu/2M/you/mmAVw7oC9mxuhRb/DscyP9DilfiszcL0VNj25XcuF5PWnTKnyz16yZn9OyWVfOb9mdlT+u5va//83nCKOvWvUqDLjuKrq0vZS25/UiPj6eiy/twTP/e4Trr/07bc/rRdq6DVzWt5ffoUZVs2u78HPqhoOv/3hpK8pUr8SL7e5kePu7WDblKwDmDv+Akd3uY2S3+/j0P+NZO2954BP6WWedzrUD+nLeed1p0qQj3bp14NRT6/gdVr6ELPIllkUtqZvZGWbW3sxKH9beJVrHjIaZMz8/2BudP38RNWoWjQkq4+PjSUxMJD4+nqSkRPbs3kNGRgarVq4BYM6sr+jes5PPUUZPmWoVqdfubBaNm3WwrclVHfj8mXch/GgB9vyy43fva9DrPJZO+qrQ4vTLGWfUZ/7XC9m7dx+hUIg5n82ld++ufoeVL1nHscSyqCR1M7uF8BjMm4ElZpazS/dINI5ZIJzjvUmvMPuzSfz5mit+t/mqfn2YPm22D4EVro3pm3lx2MssWDKD736Yzc4du5j87lSKFStGo7PPAqBHr06B/gXXcUg/Zj7yJi7r0OWz8idXoUHPllw7ZShXjLmLCnWq/uY9xRKLc2rrP7Lio68LO9xCt3TpCs6/oAUVK1YgKSmRrl3aUbtWDb/DypegJPVo1dT/CjRxzu0yszrARDOr4z3M+oj/ePHmTxgIkFg8meIJZaMUXu46d7yc9PRNJFeuxHuTx5CSspIvv5gPwB133kBmKMSE8ZOO8SknvnLlytK5WztaNOrI9u07eWnMU1xyWU+uv/bvPPDIPRQvkcDsmV8Syor1P955U69dY/b8sp2NS37ipJZnHmwvVjyBzP0ZjO75D07v0pQejw3k1UuHHtxev8M5pC1ICXzpBWDFilQef+x5PvrwDXbv3sN33y0lFDqx/zzE+KNHIxatpB7nnNsF4Jz7yczaEE7sJ3OUpJ5zPoVypU8t9BFG6embAPh5yy+8P2UaTZo04ssv5tP3ykvo3KUtF/XoV9gh+eLCNueyds16fvllKwAfTplO0+Zn8/aEKfTuFj4Hrduexyn16vgYZfTUanoa9Ts04dQ2Z1OsRAIlyiRx0dN/Y0f6r6yYGv4l/8PUBfR47LrfvO+sni1ZOjn4pZdsL78yjpdfGQfA0KH3sD4tzzdBxoQT+1fSIdGqqW8ys7OzX3gJvgeQDPwhSsfMl5IlkyhdutTB9XbtLmTZshTad2jFrYP+yhWXX8fevft8jrJwrE9Lp0nTRiQlJQJwQeuW/JiyikrJFQEoXjyBG2/7C2NfHu9nmFHz6X/H81zLm3n+gtt49+Zh/PTlMibf9gIp0xZQ59wGAJzU8kx+XX0oiZUok8RJLc8kZdo3foVd6CpXrgRA7do16N27K2+Oe9fniPKnoKYJ8Fu0eupXA5k5G5xzmYTnNxie+1v8VaVKMq+9GZ6bvlixeCZOmMKMT+aw8LuZFC9RnPcmh6dBXjB/EYNu/YefoUbdwm8W8/7kaUybPZHMzBBLvl/Oa69M4O77b6Vj59ZYXBxjR4/jiznzjv1hAfLlC1Po/cwNNB/QlQN79vHB3YeG8J3euRmr5nxPxt79PkZYuCaMf4mKlSqQmZHJLbcMZvv23184PpEEZZy6OReb91H5UX6JVSWLlfA7hJhxU7nGfocQM/6V/qnfIcSMjAPr852SnzrpqohzzqC1r8XsrwDdfCQiQnBq6krqIiJo7hcRkUAJSk1dSV1EhNgf1RIpJXURESArIAUYJXUREXShVEQkUILRT1dSFxEB1FMXEQmUTAtGX11JXUQElV9ERAIlKOUXPc5ORITwkMZIl0iYWbyZLTSz973Xdc1snpmlmtl4MyvutZfwXqd62+vk53soqYuIEC6/RLpE6FZgeY7X/wGecs7VA7YCA7z2AcBWr/0pb788U1IXEaFgH2dnZrWA7sBI77UB7YCJ3i5jgN7eei/vNd729t7+eaKkLiIChHARL2Y20MwW5FgGHvZxTwN3ceh3QCVgm/dcCYA0oKa3XhNYBwefO7Hd2z9PdKFURITju1Ca89GbhzOzHsBm59w33qM8C5WSuogI4ApuUOP5wEVm1g1IBMoCzwDlzayY1xuvBaz39l8P1AbSzKwYUA74Ja8HV/lFRISCq6k75+51ztVyztUBrgBmOueuBGYBfbzd+gOTvPXJ3mu87TNdPh5Jp6QuIkLBD2nMxd3A7WaWSrhmPsprHwVU8tpvB+7Jz/dQ+UVEhOjcUeqc+xT41FtfBTTPZZ99wKUFdUwldRERIDMgEwUoqYuIUKAXSn0Vs0l994F9focQM3QuDhmy51O/Q4gZlZLK+B1CoARl7peYTeoiIoVJPXURkQBRT11EJEBCeR8aHlOU1EVEID/jz2OKkrqICKqpi4gEimrqIiIBovKLiEiAqPwiIhIgGv0iIhIgKr+IiASILpSKiASIauoiIgGi8ouISIDk4wlyMUVJXUQECKmnLiISHCq/iIgEiMovIiIBop66iEiABGVIY5zfAYiIxIKQcxEvR2Nmtc1slpktM7OlZnar18XZqAIAAAmfSURBVF7RzKab2Y/ezwpeu5nZs2aWamaLzeyc/HwPJXUREcLll0iXY8gE/u6cawC0BG40swbAPcAM51x9YIb3GqArUN9bBgIv5Od7KKmLiFBwSd05l+6c+9Zb3wksB2oCvYAx3m5jgN7eei9grAubC5Q3s+p5/R5K6kfQuVMbli6Zw4pln3PXnTf6HY6vdC4OKern4rob+jN77hRmfzWZF0c9QYkSxTnp5Jp8NGM8cxd+zIiXnyQhIcHvMPPEORfxEikzqwM0BuYBVZ1z6d6mjUBVb70msC7H29K8tjxRUs9FXFwczz7zMD16XsUfGrXl8st7c+aZ9f0Oyxc6F4cU9XNRrXoV/nJ9Pzq36UPrcy8iLj6O3pd05/4H7mD4/8bQsnFntm3bQd+rL/E71Dw5np66mQ00swU5loGHf56ZlQbeBm5zzu3Iuc2FfzNE5cqsknoumjdrzMqVP7F69VoyMjKYMGESF/Xs7HdYvtC5OETnAuLj40lMSiQ+Pp6SSUls2rSFC1q1ZMp7HwMw4Y336Nq9g89R5o07nv+cG+Gca5pjGZHzs8wsgXBCf905947XvCm7rOL93Oy1rwdq53h7La8tT6KW1M2suZk189YbmNntZtYtWscrSDVqVmNd2oaDr9PWp1OjRjUfI/KPzsUhRf1cbEzfzAvPjebbJTNZnPIZO3bsZPHCJezYvoNQKATAhg0bqV69is+R5k3IZUW8HI2ZGTAKWO6cezLHpslAf2+9PzApR/vV3iiYlsD2HGWa4xaVcepmNoTwFd1iZjYdaAHMAu4xs8bOuYejcVwRiZ5y5cvSpXt7mv2xA9u372TkmKdp2+FCv8MqMAV4R+n5QD/gezNb5LXdBzwKTDCzAcAa4DJv24dANyAV2ANck5+DR+vmoz7A2UAJwhcEajnndpjZ44QvGOSa1L261EAAiy9HXFypKIV3dBvWb6R2rRoHX9eqWZ0NGzb6EovfdC4OKernolWbc1m7Jo1fftkKwAdTptO85TmULVeW+Ph4QqEQNWpUIz198zE+KTYV1B2lzrnPATvC5va57O+AArvqHq3yS6ZzLuSc2wOszL5I4Jzby1EeMJKzTuVXQgeYv2AR9erVpU6d2iQkJHDZZb2Y8v403+Lxk87FIUX9XKxfl845TRuRlJQIwIWtzyVlxUq++GwePXuHry1c1rc3Uz+c4WeYeXY8NfVYFq2e+gEzK+kl9SbZjWZWjhPgqVGhUIhbb7ufDz94g/i4OF4ZM55ly1L8DssXOheHFPVz8e03i3l/0jSmz3mHUGYm3y9ezquvjOeTaZ8yfPST3HP/rXy/eDlvjJ3od6h5khWQCb0sGjOTmVkJ59z+XNqTgerOue+P9RnFitcMxhkWiZJKSWX8DiFmbNq+4kjljoidVbVFxDln6aZ5+T5etESlp55bQvfafwZ+jsYxRUTy41ijWk4UmqVRRITglF+U1EVECM7Uu0rqIiKopy4iEijqqYuIBEjIhfwOoUAoqYuIoAdPi4gEih48LSISIOqpi4gEiEa/iIgEiEa/iIgEiKYJEBEJENXURUQCRDV1EZEAUU9dRCRANE5dRCRA1FMXEQkQjX4REQkQXSgVEQmQoJRf4vwOQEQkFrjj+O9YzKyLmf1gZqlmdk8hhH+QeuoiIhRcT93M4oHngY5AGjDfzCY755YVyAGOQUldRIQCrak3B1Kdc6sAzGwc0Aso2kk988B68zsGADMb6Jwb4XccsUDn4hCdi0OCci6OJ+eY2UBgYI6mETnOQU1gXY5taUCL/EcYGdXUj23gsXcpMnQuDtG5OKTInQvn3AjnXNMcS8z8UlNSFxEpWOuB2jle1/LaCoWSuohIwZoP1DezumZWHLgCmFxYB4/ZmnoMiZl/VsUAnYtDdC4O0bnIwTmXaWY3AR8D8cBo59zSwjq+BWXAvYiIqPwiIhIoSuoiIgGipH4Eft7mG2vMbLSZbTazJX7H4iczq21ms8xsmZktNbNb/Y7JL2aWaGZfm9l33rl4wO+YJEw19Vx4t/mmkOM2X+BPhXWbb6wxs1bALmCsc66h3/H4xcyqA9Wdc9+aWRngG6B3UfxzYWYGlHLO7TKzBOBz4Fbn3FyfQyvy1FPP3cHbfJ1zB4Ds23yLJOfcHOBXv+Pwm3Mu3Tn3rbe+E1hO+O7BIseF7fJeJniLeogxQEk9d7nd5lsk//JK7sysDtAYmOdvJP4xs3gzWwRsBqY754rsuYglSuoix8nMSgNvA7c553b4HY9fnHMh59zZhO+YbG5mRbY0F0uU1HPn622+Eru8+vHbwOvOuXf8jicWOOe2AbOALn7HIkrqR+Lrbb4Sm7yLg6OA5c65J/2Ox09mVtnMynvrSYQHFazwNyoBJfVcOecygezbfJcDEwrzNt9YY2ZvAl8Bp5tZmpkN8Dsmn5wP9APamdkib+nmd1A+qQ7MMrPFhDtB051z7/sck6AhjSIigaKeuohIgCipi4gEiJK6iEiAKKmLiASIkrqISIAoqUtUmFnIG/K3xMzeMrOS+fisV8ysj7c+0swaHGXfNmZ2Xh6O8ZOZJec1RpFYoaQu0bLXOXe2N6vjAeD6nBvNLE+PUnTO/eUYsyK2AY47qYsEhZK6FIbPgHpeL/ozM5sMLPMmhHrMzOab2WIzuw7Cd26a2TBvPvtPgCrZH2Rmn5pZU2+9i5l9683pPcObZOt6YJD3r4QLvTsf3/aOMd/MzvfeW8nMpnlzgY8ErHBPiUh06MHTElVej7wrMNVrOgdo6JxbbWYDge3OuWZmVgL4wsymEZ798HSgAVAVWAaMPuxzKwMvAa28z6ronPvVzF4EdjnnHvf2ewN4yjn3uZmdRPgu4TOBIcDnzrkHzaw7UFTvkpWAUVKXaEnypmWFcE99FOGyyNfOudVeeyfgj9n1cqAcUB9oBbzpnAsBG8xsZi6f3xKYk/1ZzrkjzffeAWgQnrYFgLLeLIutgP/z3vuBmW3N4/cUiSlK6hIte71pWQ/yEuvunE3Azc65jw/bryDnU4kDWjrn9uUSi0jgqKYufvoY+Js3nS1mdpqZlQLmAJd7NffqQNtc3jsXaGVmdb33VvTadwJlcuw3Dbg5+4WZZf+imQP09dq6AhUK7FuJ+EhJXfw0knC9/FvvodbDCf/r8V3gR2/bWMIzRP6Gc24LMBB4x8y+A8Z7m6YAF2dfKAVuAZp6F2KXcWgUzgOEfyksJVyGWRul7yhSqDRLo4hIgKinLiISIErqIiIBoqQuIhIgSuoiIgGipC4iEiBK6iIiAaKkLiISIP8PNCj6TPvJoCYAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GwPv8x0uWuQu"
      },
      "source": [
        "# Model testing with user input data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2nTj5SU3srBV",
        "outputId": "d044b598-fa56-4e97-d64a-8d871c419f5a"
      },
      "source": [
        "# Load from file\n",
        "from sklearn.externals import joblib\n",
        "joblib_lgr = joblib.load('/content/joblib_lgr.pkl')\n",
        "joblib_lgr"
      ],
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression(C=2.0, class_weight=None, dual=False, fit_intercept=True,\n",
              "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
              "                   multi_class='ovr', n_jobs=None, penalty='l2',\n",
              "                   random_state=None, solver='liblinear', tol=0.0001, verbose=0,\n",
              "                   warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WnZL4XELmd_4"
      },
      "source": [
        "tfidf_1 = joblib.load(\"/content/tfidf.pickle\")\n",
        "scaler_2 = joblib.load('/content/scaler.pickle')"
      ],
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xkXVzCmesn03"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "# pipeline\n",
        "texts = \"แล้วมีสาขาไหนบ้างโปรนี้\"\n",
        "# create df\n",
        "ui = pd.DataFrame({\"texts\": texts}, index=[0])\n",
        "# preprocess\n",
        "ui[\"processed\"] = \"|\".join(process_text(texts))\n",
        "ui[\"wc\"] =  ui.processed.map(lambda x: len(x.split(\"|\")))\n",
        "ui[\"uwc\"] = ui.processed.map(lambda x: len(set(x.split(\"|\"))))\n",
        "# transform\n",
        "tfidf = tfidf_1.transform(ui[\"texts\"])\n",
        "scaler = scaler_2.transform(ui[[\"wc\",\"uwc\"]].astype(float))\n",
        "tfidf_user = pd.DataFrame(tfidf.toarray())\n",
        "scaler_user = pd.DataFrame(scaler)\n",
        "user_texts = pd.concat([scaler_user, tfidf_user], axis=1)"
      ],
      "execution_count": 65,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 126
        },
        "id": "Fsl_4VmVp3Xx",
        "outputId": "07dd078b-07a1-4f4e-9f47-cfa1724d5d3e"
      },
      "source": [
        "user_texts"
      ],
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>10</th>\n",
              "      <th>11</th>\n",
              "      <th>12</th>\n",
              "      <th>13</th>\n",
              "      <th>14</th>\n",
              "      <th>15</th>\n",
              "      <th>16</th>\n",
              "      <th>17</th>\n",
              "      <th>18</th>\n",
              "      <th>19</th>\n",
              "      <th>20</th>\n",
              "      <th>21</th>\n",
              "      <th>22</th>\n",
              "      <th>23</th>\n",
              "      <th>24</th>\n",
              "      <th>25</th>\n",
              "      <th>26</th>\n",
              "      <th>27</th>\n",
              "      <th>28</th>\n",
              "      <th>29</th>\n",
              "      <th>30</th>\n",
              "      <th>31</th>\n",
              "      <th>32</th>\n",
              "      <th>33</th>\n",
              "      <th>34</th>\n",
              "      <th>35</th>\n",
              "      <th>36</th>\n",
              "      <th>37</th>\n",
              "      <th>...</th>\n",
              "      <th>6117</th>\n",
              "      <th>6118</th>\n",
              "      <th>6119</th>\n",
              "      <th>6120</th>\n",
              "      <th>6121</th>\n",
              "      <th>6122</th>\n",
              "      <th>6123</th>\n",
              "      <th>6124</th>\n",
              "      <th>6125</th>\n",
              "      <th>6126</th>\n",
              "      <th>6127</th>\n",
              "      <th>6128</th>\n",
              "      <th>6129</th>\n",
              "      <th>6130</th>\n",
              "      <th>6131</th>\n",
              "      <th>6132</th>\n",
              "      <th>6133</th>\n",
              "      <th>6134</th>\n",
              "      <th>6135</th>\n",
              "      <th>6136</th>\n",
              "      <th>6137</th>\n",
              "      <th>6138</th>\n",
              "      <th>6139</th>\n",
              "      <th>6140</th>\n",
              "      <th>6141</th>\n",
              "      <th>6142</th>\n",
              "      <th>6143</th>\n",
              "      <th>6144</th>\n",
              "      <th>6145</th>\n",
              "      <th>6146</th>\n",
              "      <th>6147</th>\n",
              "      <th>6148</th>\n",
              "      <th>6149</th>\n",
              "      <th>6150</th>\n",
              "      <th>6151</th>\n",
              "      <th>6152</th>\n",
              "      <th>6153</th>\n",
              "      <th>6154</th>\n",
              "      <th>6155</th>\n",
              "      <th>6156</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>-0.44465</td>\n",
              "      <td>-0.498125</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>1 rows × 6159 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "      0         1     0     1     2     ...  6152  6153  6154  6155  6156\n",
              "0 -0.44465 -0.498125   0.0   0.0   0.0  ...   0.0   0.0   0.0   0.0   0.0\n",
              "\n",
              "[1 rows x 6159 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AQvjXbzKYwJ5",
        "outputId": "18e1ec7c-5558-46c7-9170-64c7303e774d"
      },
      "source": [
        "# neu 1, neg 0, q 3, pos 2\n",
        "p = joblib_lgr.predict_proba(user_texts)\n",
        "p"
      ],
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.02429741, 0.04469512, 0.01210147, 0.91890601]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 67
        }
      ]
    }
  ]
}