notebooks/Sequence_manager_explanation/Seq_manager_explanation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sequence manager explanation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Presentation\n",
    "\n",
    "As presented in KidlearnStarter notebook, the Kidlearn library includes a set of algorithms that we develop to guide pedagogical activities. We introduce two different algorithms ZPDES that uses a simple graph of knowledge and RiARiT that includes a more complex student model. The details of the algorithms and the user studies are presented in the following paper: B. Clement, D. Roy, P.-Y. Oudeyer, M. Lopes, Multi-Armed Bandits for Intelligent Tutoring Systems, Journal of Educational Data Mining (JEDM), 2015 [(arXiv:1310.3174 [cs.AI])](http://arxiv.org/abs/1310.3174).\n",
    "\n",
    "These algorithms are implemented in the ***seq_manager*** module of the library, with some others sequence managers used to evaluate our algorithms. In this notebook, we will describe how to define the **activity** for each algorithms and what are the files used to define the algorithms parameters. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import kidlearn_lib as k_lib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lexical details\n",
    "\n",
    "Vocabulary is important to us to describe our problems and objects that we will use, so before beginning our explanation, here is an explanation of the vocabulary that will be used in this notebook.\n",
    "\n",
    "We will speak about **activity parameters** (or **actions**). These parameters are used to define an **activity**. For example, the difficulty level, the type of exercise, the contents of an exercise or the presentation of this content. \n",
    "\n",
    "These parameters can take different **parameter's values** (or **values**), for example the difficulty level can have a scale and you will choose level 1 or 2 or higher depending of the level of the learner. This values can be ordered, like a difficulty level, wich means all values will not be available all the time. Or there can be no order, then all values can be available all time. \n",
    "\n",
    "So to instantiate an **exercise**, one value will be chosen for each parameters needed in the current activity. An activity is a set of all possible exercises which can be defined with a set of parameters.\n",
    "\n",
    "The parameters can be regroup in **group of parameter**. And these groups can be **hierarchically** organised. This mean the use of some paramameters will depend of other parameters. For example, if one parameter dertermine the type of exercise, some parameters will be used and others not, depending of the type of exercise chosen. In any case there is always a **main group of paramaters** whose values are chosen first and determine the rest of the parameters choise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# The sequence manager objects\n",
    "\n",
    "There are 5 different sequence managers which can be used : \n",
    "- RiARiT : an algorithm based on student skill estimation\n",
    "- ZPDES : an algorithm based on the success of the student\n",
    "- Sequence : a predefined sequence of activities (defined by an expert)\n",
    "- POMDP : an algorithm based on a probabilistic model to describe the activity learning.\n",
    "- Random: a random sequence of all activities. \n",
    "\n",
    "## Some general information about activity description : \n",
    "\n",
    "### For load files\n",
    "\n",
    "\"h\" : give information about the ordered nature of the values for each actions.  \n",
    "\n",
    "\"nb_stay\" : correspond to the number of activities which have to be done for each didactic parameter, some time it's necessarry to stay with the same parameter value for a number of step for pedagogical or didactic reasons. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ZPDES : Zone of Proximal Development and Empirical Success\n",
    "\n",
    "### To describe the activity\n",
    "\n",
    "[zpdes_graph_example.json](graph/zpdes_graph_example.json)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'KC': ['S0', 'S1', 'S2'],\n",
       " 'N2': {'actions': ['Act1', 'Act2', 'Act3'],\n",
       "  'h': [1, 0, 0],\n",
       "  'nb_stay': [2, 1, 1],\n",
       "  'ssbg': [['V11', 'V12', 'V13'], ['V21', 'V22'], ['V31', 'V32']]},\n",
       " 'V13': {'ssbg': [['V131', 'V132']]},\n",
       " 'act_prime': 'N2'}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{\n",
    "    \"act_prime\" : \"N2\",      # What is the main group of paramater   \n",
    "    \"KC\":[\"S0\",\"S1\",\"S2\"],   # Optional, to define skill name \n",
    "\n",
    "    \"N2\":{                                # Description of the first/main parameters group    \n",
    "        \"actions\":[\"Act1\",\"Act2\",\"Act3\"], # Optional Name of the parameters\n",
    "        \n",
    "        \"h\":[1,0,0],                      # Optional : ordered hierarchically : 1\n",
    "                                          # by default not ordered : 0\n",
    "        \n",
    "        \"nb_stay\":[2,1,1],                # Optional, Definition of the number of staying step              \n",
    "        \"ssbg\":[[\"V11\", \"V12\", \"V13\"],\n",
    "                [\"V21\", \"V22\"], \n",
    "                [\"V31\", \"V32\"]]\n",
    "    },\n",
    "    \n",
    "    # If we cant to add a dependant group it is enough to just add the name fo the value: \n",
    "    # Not in the file : For example if the value V13 lead to another group of paramaters :\n",
    "    \n",
    "    \"V13\" : {\n",
    "        \"ssbg\":[[\"V131\",\"V132\"]]\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To define algorithm paramaters\n",
    "\n",
    "[ZPDES.json](params_files/ZPDES.json)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ZpdesSsbg': {'ZpdesSsb': {'deactZPDval': 0.8,\n",
       "   'filter1': 0.2,\n",
       "   'filter2': 0.8,\n",
       "   'h_promote_coeff': 1,\n",
       "   'promote_coeff': 1,\n",
       "   'stepUpdate': 8,\n",
       "   'thresHierarProm': 0.3,\n",
       "   'uniformval': 0.05,\n",
       "   'upZPDval': 0.5}},\n",
       " 'algo_name': 'ZpdesHssbg',\n",
       " 'graph': {'file': 'zpdes_graph_example', 'main_act': 'N2', 'path': 'graph/'}}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{\n",
    "    \"algo_name\" : \"ZpdesHssbg\",  # First name of the algorithm \n",
    "    \"graph\": {             # Here you define the activity paramaters\n",
    "        \n",
    "        \"file_name\" :\"zpdes_graph_example\", # name of the activity description file\n",
    "        \"path\" : \"graph/\",            # path to the file\n",
    "        \"main_act\":\"N2\"              # name of the main action\n",
    "        },\n",
    "    \n",
    "# ZpdesSsbg : name of a subpart of the algorithm, wich follow correspond\n",
    "#              to the paramaters fo this particular part\n",
    "    \"ZpdesSsbg\": {\n",
    "        \n",
    "# ZpdesSsb : name of a subpart of the algorithm ...\n",
    "        \"ZpdesSsb\" :{\n",
    "            \n",
    "            \"filter1\": 0.2,  # Filster1/2 are the paramaters that manage the bandits values \n",
    "            \"filter2\": 0.8, # valBandit = f1 * valBandit + f2 * reward\n",
    "            \n",
    "            \"uniformval\": 0.05, # initialisation value for the first activated bandits\n",
    "            \"stepUpdate\" : 8,   # number of sucess to take into account to calcul\n",
    "                                # the reward of the algorithm\n",
    "            \n",
    "            \"upZPDval\" : 0.5,    # successrate threshold to activate a new value for ordered parameters \n",
    "            \"deactZPDval\" : 0.8, #success rate threshold to deactivate values for ordered parameters\n",
    "            \"promote_coeff\" : 1,    # Optional, default 1, when an ordered value is activated, \n",
    "                                    # his bandit value depend of already activated values followin :\n",
    "                                    # newBandit = min(activatedBandits) * promot_coeff\n",
    "            \n",
    "            \"thresHierarProm\" : 0.3, # thereshold to activate bandit when a groups of parameters \n",
    "                                     # is hierarchical and call other groups of parameters\n",
    "            \n",
    "            \"h_promote_coeff\" : 1   # Optional, default 1, when a hierarchical value is activated, \n",
    "                                    # his bandit value depend of already activated values followin :\n",
    "                                    # newBandit = min(activatedBandits) * promot_coeff\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a ZPDES sequence manager"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "activity to do : {u'N2': [0, 0, 1]}\n"
     ]
    }
   ],
   "source": [
    "zpdes = k_lib.config.seq_manager(params_file=\"ZPDES\",directory=\"params_files\")\n",
    "\n",
    "# to sample zpdes bandit and have an activity to do : \n",
    "act = zpdes.sample()\n",
    "print \"activity to do : {}\".format(act)\n",
    "\n",
    "# To update zpdes with result\n",
    "zpdes.update(act,result=0.5)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RiARiT : The Right Activity at the Right Time\n",
    "\n",
    "### To describe the activity\n",
    "\n",
    "RiARiT use tables to define the relation between skills and activities. \n",
    "\n",
    "Two ways exist to define the tables : \n",
    "- with a text-file, there is an example in **graph/N2.txt**.\n",
    "- in json, there is an example in **graph/N2.json**.\n",
    "\n",
    "\n",
    "#### Text file \n",
    "\n",
    "[N2.txt](graph/N2.txt) example : \n",
    "\n",
    "**Competences : S0, S1, S2**  <-  here we define the skill used in the activities \n",
    " (S0 could be addition skill for example)\n",
    "\n",
    "**Actions : Act1, Act2, Act3** <- here we define the didactics parameters used \n",
    " (Act1 could be difficulty level)\n",
    "\n",
    "**Nb_stay : 2, 1, 1 ** <- here we define the number of activities which have to be done for each didactic parameter, for act 1 there will be 2 activities with the same parameter value chosen, and for the other the parameter value will be chosen at each step\n",
    "\n",
    "**1 | 0 | V1 | 0.2   0.1   0  | 0   0   0  | 0   0   1 **<-- example : first line of the table \n",
    "\n",
    " The first value represents the number of the didactic parameter: 1 (the first parameter)\n",
    " The second value : 0 (1 else) say that this value is not used (here) to call another group of parameters, if the value is 1 that means another group of parameters is used to define the exercises.  \n",
    " The second value : V1 is a named gave to the parameter's value (number 1 here). To load another parameter group using a value, the value must have the same name than the group of the parameter file.  \n",
    "\n",
    " The next 3 numbers (0.2, 0.1, 0) represent the minimum required competence level, for each KC, to be able to solve that exercise\n",
    "\n",
    " The next 3 numbers (0, 0, 0) represent the activation threshold for this parameter's value depending on the learner skill level. Here, all values are null, so this parameter's value is activated from the beginning. \n",
    "\n",
    " The next 3 numbers (0.2, 0.1, 0) represent the deactivation threshold for this parameter's value depending on the learner skill level. For this example, there is one value equal to 1, so this parameter's value will always be activated until the student has totally mastered the third skill. \n",
    " \n",
    " #### JSON File \n",
    "\n",
    "[N2.json](graph/N2.json) example : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'action': ['Act1', 'Act2', 'Act3'],\n",
       " 'competencies': ['S0', 'S1', 'S2'],\n",
       " 'h': [1, 0, 0],\n",
       " 'nb_stay': [2, 1, 1],\n",
       " 'table': {'Act1': {'V1': {'deacti': [1, 1, 1],\n",
       "    'hierarchical': 0,\n",
       "    'impact': [0.2, 0.1, 0],\n",
       "    'requir': [0, 0, 0]}}}}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{\n",
    "    \"competencies\": [\"S0\",\"S1\",\"S2\"], # Definition of the competencies\n",
    "    \n",
    "    \"action\": [\"Act1\",\"Act2\",\"Act3\"], # Definition of the didactic parameters\n",
    "    \n",
    "    \"nb_stay\": [2,  1,  1], # Definition of the number of staying step\n",
    "    \n",
    "    \"h\" : [1, 0, 0],      # Optional : ordered hierarchically : 1, 0 otherwise\n",
    "    \n",
    "    \"table\": {\n",
    "        \"Act1\": {  # For the first parameter\n",
    "            \n",
    "            \"V1\": {  # First value's name\n",
    "                \n",
    "                \"hierarchical\" : 0, # Is the value use others parameters \n",
    "                \"impact\": [0.2,  0.1,0], # the minimum required competence level\n",
    "                \"requir\": [0,    0,  0], # required level \n",
    "                \"deacti\": [1,    1,  1]  # deactivation threshold\n",
    "            }\n",
    "            # next in the file N2.json\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To define algorithm paramaters\n",
    "\n",
    "[RIARIT.json](params_files/RIARIT.json) example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'RT': {'file_name': 'N2', 'main_act': 'N2', 'path': 'RT/'},\n",
       " 'RiaritSsbg': {'RiaritSsb': {'filter1': 0.2,\n",
       "   'filter2': 0.8,\n",
       "   'uniformval': 0.05},\n",
       "  'levelupdate': 0.6},\n",
       " 'name': 'RiaritHssbg'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "{\n",
    "    \"name\" : \"RiaritHssbg\", # First name of the algorithm \n",
    "    \n",
    "    \"RT\": {                 # Here you define the activity paramaters\n",
    "        \n",
    "        \"file_name\":\"N2\",        # name of the activity description file (describe before)\n",
    "        \"path\" : \"RT/\",     # path to the file\n",
    "        \"main_act\":\"N2\"     # name of the main action (same that the name file here)\n",
    "        },\n",
    "    \n",
    "# RiaritSsbg : name of a subpart of the algorithm, wich follow correspond\n",
    "#              to the paramaters fo this particular part\n",
    "    \"RiaritSsbg\": {         \n",
    "        \"levelupdate\": 0.6, # RiaRiT parameters witch update the estimated level of the \n",
    "                            # student if he succeed the activity\n",
    "        \n",
    "# RiaritSsb : name of a subpart of the algorithm \n",
    "        \"RiaritSsb\": {\n",
    "            \"filter1\": 0.2,    # Filster1/2 are the paramaters that manage the bandits values \n",
    "            \"filter2\": 0.8,    # valBandit = f1 * valBandit + f2 * reward\n",
    "            \n",
    "            \"uniformval\": 0.05 # initialisation value for the first activated bandits\n",
    "        }\n",
    "    }\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a riarit sequence manager : "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "riarit = k_lib.config.seq_manager(params_file=\"RIARIT\",directory=\"params_files\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sequence : predefined by an expert (TOFINISH)\n",
    "\n",
    "The predefined sequence can be define in a json file. The is an example in **sequence_def/example_seq.json**. The predefined sequence works with group or exercises. When a learner is working with a sequence, he will begin with the first exercise of the first group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "seq = {\n",
    "    \"activity\": {\n",
    "\n",
    "        \"group0\": [\n",
    "            {\"N2\" : [0,0,0]},\n",
    "            {\"N2\" : [0,1,0]},\n",
    "            {\"N2\" : [0,0,1]}],\n",
    "            \n",
    "        \"group1\" : [\n",
    "            {\"N2\" : [1,0,0]},\n",
    "            {\"N2\" : [1,1,0]},\n",
    "            {\"N2\" : [1,0,1]},\n",
    "            {\"N2\" : [1,1,1]}],\n",
    "\n",
    "        \"group2\" : [\n",
    "            {\"N2\" : [2,0,0]},\n",
    "            {\"N2\" : [2,1,0]},\n",
    "            {\"N2\" : [2,0,1]},\n",
    "            {\"N2\" : [2,1,1]}]\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Random (TODO)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}