3. train-hyperparameter-tune-deploy-with-tensorflow.ipynb

{
  "cells": [
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Copyright (c) Microsoft Corporation. All rights reserved.\n\nLicensed under the MIT License."
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "bf74d2e9-2708-49b1-934b-e0ede342f475"
        }
      },
      "cell_type": "markdown",
      "source": "# 1. Training, hyperparameter tune, and deploy with TensorFlow\n\n## Introduction\nThis tutorial shows how to train a simple deep neural network using the MNIST dataset and TensorFlow on Azure Machine Learning. MNIST is a popular dataset consisting of 70,000 grayscale images. Each image is a handwritten digit of `28x28` pixels, representing number from 0 to 9. The goal is to create a multi-class classifier to identify the digit each image represents, and deploy it as a web service in Azure.\n\nFor more information about the MNIST dataset, please visit [Yan LeCun's website](http://yann.lecun.com/exdb/mnist/).\n\n"
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "c377ea0c-0cd9-4345-9be2-e20fb29c94c3"
        },
        "trusted": true
      },
      "cell_type": "code",
      "source": "%matplotlib inline\nimport numpy as np\nimport os\nimport matplotlib\nimport matplotlib.pyplot as plt",
      "execution_count": 5,
      "outputs": []
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "edaa7f2f-2439-4148-b57a-8c794c0945ec"
        },
        "trusted": true
      },
      "cell_type": "code",
      "source": "import azureml\nfrom azureml.core import Workspace, Run\n\n# check core SDK version number\nprint(\"Azure ML SDK Version: \", azureml.core.VERSION)",
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Azure ML SDK Version:  1.0.2\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Diagnostics\nOpt-in diagnostics for better experience, quality, and security of future releases."
    },
    {
      "metadata": {
        "tags": [
          "Diagnostics"
        ],
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.telemetry import set_diagnostics_collection\n\nset_diagnostics_collection(send_diagnostics=True)",
      "execution_count": 7,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Initialize workspace\nInitialize a [Workspace](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture#workspace) object from the existing workspace you created in the Prerequisites step. `Workspace.from_config()` creates a workspace object from the details stored in `config.json`."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core.workspace import Workspace\n\nws = Workspace.from_config()\nprint('Workspace name: ' + ws.name, \n      'Azure region: ' + ws.location, \n      'Subscription id: ' + ws.subscription_id, \n      'Resource group: ' + ws.resource_group, sep = '\\n')",
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Found the config file in: /home/nbuser/library/aml_config/config.json\nWorkspace name: AMLSworkspace\nAzure region: westeurope\nSubscription id: 70b8f39e-8863-49f7-b6ba-34a80799550c\nResource group: resgrpAMLS\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "59f52294-4a25-4c92-bab8-3b07f0f44d15"
        }
      },
      "cell_type": "markdown",
      "source": "## Create an Azure ML experiment\nLet's create an experiment named \"tf-mnist\" and a folder to hold the training scripts. The script runs will be recorded under the experiment in Azure."
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "bc70f780-c240-4779-96f3-bc5ef9a37d59"
        },
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core import Experiment\n\nscript_folder = './tf-mnist'\nos.makedirs(script_folder, exist_ok=True)\n\nexp = Experiment(workspace=ws, name='tf-mnist')",
      "execution_count": 9,
      "outputs": []
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "defe921f-8097-44c3-8336-8af6700804a7"
        }
      },
      "cell_type": "markdown",
      "source": "## Download MNIST dataset\nIn order to train on the MNIST dataset we will first need to download it from Yan LeCun's web site directly and save them in a `data` folder locally."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import os\nimport urllib\n\nos.makedirs('./data/mnist', exist_ok=True)\n\nurllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz', filename = './data/mnist/train-images.gz')\nurllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz', filename = './data/mnist/train-labels.gz')\nurllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz', filename = './data/mnist/test-images.gz')\nurllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz', filename = './data/mnist/test-labels.gz')",
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 10,
          "data": {
            "text/plain": "('./data/mnist/test-labels.gz', <http.client.HTTPMessage at 0x7fa22ad39748>)"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "c3f2f57c-7454-4d3e-b38d-b0946cf066ea"
        }
      },
      "cell_type": "markdown",
      "source": "## Show some sample images\nLet's load the downloaded compressed file into numpy arrays using some utility functions included in the `utils.py` library file from the current folder. Then we use `matplotlib` to plot 30 random images from the dataset along with their labels."
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "396d478b-34aa-4afa-9898-cdce8222a516"
        },
        "trusted": true
      },
      "cell_type": "code",
      "source": "from utils import load_data\n\n# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the neural network converge faster.\nX_train = load_data('./data/mnist/train-images.gz', False) / 255.0\ny_train = load_data('./data/mnist/train-labels.gz', True).reshape(-1)\n\nX_test = load_data('./data/mnist/test-images.gz', False) / 255.0\ny_test = load_data('./data/mnist/test-labels.gz', True).reshape(-1)\n\ncount = 0\nsample_size = 30\nplt.figure(figsize = (16, 6))\nfor i in np.random.permutation(X_train.shape[0])[:sample_size]:\n    count = count + 1\n    plt.subplot(1, sample_size, count)\n    plt.axhline('')\n    plt.axvline('')\n    plt.text(x = 10, y = -10, s = y_train[i], fontsize = 18)\n    plt.imshow(X_train[i].reshape(28, 28), cmap = plt.cm.Greys)\nplt.show()",
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA5IAAABBCAYAAACjM5sOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJztnXdYFNf6x79nll4E0YBgQ41Ygtca9GrsepXoVS+2qKjY+BkVY4maGGOwxpio126Mxl6iaBL7taJi1MSaqIDl2jDYkc4Cy/v7Y3fnsrC7bJnZNXI+zzMP7OzsfM+ZM+ed85455z2MiMDhcDgcDofD4XA4HI6pCPZOAIfD4XA4HA6Hw+Fw/lpwR5LD4XA4HA6Hw+FwOGbBHUkOh8PhcDgcDofD4ZgFdyQ5HA6Hw+FwOBwOh2MW3JHkcDgcDofD4XA4HI5ZcEeSw+FwOBwOh8PhcDhmwR1JDofD4XA4HA6Hw+GYhWyOJGNMYIyNZ4wlMMZyGGMPGWMLGGPucmkW0vZhjH3DGLut0X7GGDvBGGspk140Y4yMbHly6BpIixtj7K5Gd5mMOn6MsVWacs1ljD1gjC1mjHnLpanRNXSNM2TUtFv5MsY8GGNTGWN/MMbSGWPPGWO/MMYiGGNMJs1ajLEtjLF4xlgqYyxLU48XMsb85dAsol8q6q89bGRpK1uNpj1sRhBjbCZj7Jwmj+mMsSuMsc/exGegvXTt/ey1Q37tVn8ZY58yxnYyxv6rubb35NTTaNrLNttL1252wx52Uk8aZG+/2qsO2atsbXEvO0iRUAMsAjAWwI8AFgCoo/nckDHWgYgK5BBljFUFEAvAA8BaADcBeAH4G4CKcmgC2A3gtp79fwMwCcBemXT1MRNAeTkFGGO+AM4DCADwLYBrAIIBfAigFWOsBRFlyZiE0wBWF9knZ4PBLuXLGBMAHATQHMAGAEsBuAHoB2Ad1HVqigzSlQD4Q113kwDkA6gHIBLAB4yxBkT0VAbd0lZ/7WEjS1vZarG1zRgKYDSAPQC2aLTaApgNoA9jrBkRZcshbK/rXMrqrr3ya7f6C2AugJcALgGQtcO4EPYqX3vp2s1uaLC1nSyK7O1X2K8O2ats5b+XiUjyDcA7AAoA7CqyPwoAAegvh65G4zSAhwD85dIwIy3favLbxUZ6jaCuFBM0ustk0vm35vz9iuzvp9k/TcY8EoD19i5bW5QvgL9rzr+oyH4nAP8F8MrG+e2tSc9kGTVKRf21p40sbWVrD5sBoAkALz37Z2vSM+YNvM6lou6+pvm1Rf2tXuj/awDuvcnlaw9dO9sNu7atbNV+NaIvax2yZ9kaSI9k97JcQ1v7AWBQOxyF+Q5AFoBwOUQZY60AvAdgPhElM8YcGWNucmiZkBY3AB8AeATgkA30FFBf30NQ90DISVsA2QC2F9n/A4AcAENk1gdjzIkx5iG3jhF9W5RvGc3fPwvvJKJcAM8BZMqka4j7mr9l5Th5Kau/drGRRnjjy9aWNoOILhBRqp6vftD8DZZD117X+XUo30Jpkd02v0751SBr/QUAIvqvXOc2B1u3rWypay+7URh7tK1s3H41hKx16HUoWy1S38tyOZLvQt3b/mvhnUSUA+CK5ns5eF/z9wFjbC/Uzk4mY+wmY8zWDbM+UDsC64hIZQO98QBqAxhjAy1nADmk6dbQQuqheNkAqjPG5Bye0AvqxnY6Y+wpY2wpY8xLRj192KJ8fwXwCsBkxlhvxlgVzfj+LwE0BhAtky4AgDHmwhgrzxirxBj7B9Q9WABwQCbJ0lR/7WUjAZTKsn0dbAagHlYFAE9kOr+9rrO9y7cwtrDNds2vHerv64St21b21gXktxta7GUnbdl+BfBa1SFblW1hpL2XZXpl+geAJwa+2wH161QnGXR/1Jz7KYAzAAZAPS75mmb/EBu+Nj4NdUOxmg20qkH9dmqK5nMg5B3auktz/gZF9jfQ7CcAjWTSPg/gYwA9AAyC+q0oAfgdgMebVr4AWgJILHRdCUAagB42yOOYIrp3AQyQUa/U1F972cjSWLavkc1QADgL9dyYWm/SdS5Ndfd1yK+t668efbsNbbXVs/c10pXdbmh07GInYeP2ayFdu9YhW5atHl1J72W5EnkHwAMD323UFJq3DLpHNee+g0KNMKhfVacASAYg2KCQamnScdRGN8UhjWF31HyWtSJC7dyooA4u8D6AKgBCoXZ4cjXa79ki75r0TNVofmYjPZuVL4CGUDvuXwP4F4BhUAc7yALQUWbtSgA6aB4sMwBcBjBORr1SU3/tZSNLe9kW0rWpzdBoLtVofvqmXefXpXxtZZvtnV9b1189+nZxJG357H0ddDXastsNI9qy20nYuP1aSNeudcheZSvHvSxXQu31RnKv5tyz9Xy3QfNdHRsU1HyN1gc20AqHumfhvUL7ZK+IUE9MTsb/enPyAayCenw7Afib3HkvlBZHAEoAv9hIzyblC3UksWwAI4vsd4M62M49AAobXue/aa6zLEavNNVfe9nI0l62hTRtbTNmafL4rcw6drnOr0v52tA2vxb5LaQpa/3Vo2cvR9JmbavXRNcmdsOIvqx2EnZqvxpIi63rkF3KVo57Wa45kn8CKM8Yc9bzXUUAz0kdMERqkjR/H+v5LlnzV7bJ6ADAGHOAeljAS6iHv8ip5QxgIdRjuh8zxt5mjL0NoKrmEC/NPslDdRPRTqh7dBoCaAUggIhGavblQ3+4YVkgojxo7jm5tWxZvlDPG3ABsLPwTlIvrbIf6nIOlDkNhXV/h7rXbpRMEqWp/trLRuqlNJRtYWxsM6IBTIN6yZ6RMsvZ6zrbvXxtbJvtnt/C2KD+2h0bl+/roBsN29kNvchpJ+3ZftWHLeuQvcpWrntZLkfyN825QwrvZIy5QD2P7oJMutrAFZX0fKfdJ9caS1r+CcAPwCYiUsqs5QrgLQBdANwqtMVqvg/XfB4uhzgRqYjoChGdJqKnjLEKUDuWJ0nedSR10NxXlWCbycq2LF/tWmQKPd85FPlrK1wB+Mh07tJUf+1lI43xppetiK1sBmPsCwBfQD1ceThpuoRlxF7X+XUoX1va5tchv0WRs/6+DtiyfO2qawe7YSgdctpJu7ZfjaRJ1jpk57KV516W6dVpPRhfIy1cJt2yUAciSUKhycFQLz6aAeCmDV4b79PksZ4NtByhjrJVdPtQk4aDms9BNkiLAPWQvAIAbWXSKGdg/9eQeQ0tO5XvIn35gnox6D+h7lVykEG3goH9baGeG3tMpvyWmvprRxtZqsrWnjYDwHSNxkbYaP6nHa9zqam7dr7Odqm/evRsPrTVluVrT1072Q2b20nYqf1qzzpkj7Itoi/Lvcw0J5ccxthSqKMi/Qj1q+s6AMZCHeGsHamXipBDNxLqML7XAXwP9eLtH0Jt4LsS0WE5dDXaAQAeALhIRE3l0jEhHYFQR6BaTkSSh1PWrDH0K9RlexeAF9Tr4jWGelL2XKk1NbqLADQDcALq6+wBdbCftlBHHGtLRNlyaGv0bVq+jLGqUAfWKQtgC9R1xwfACKiHtI4mohUy6P4IdX05DvXaSi5Ql+0HUAf5aUNEV6TW1WiXmvprDxtZ2srWXjaDMTYawDKN5udQdxoU5gkRHZFaV6NtlzpUmuquRtMe97M96+9A/G/YYRTU+V2g+XyfiDbJoavRtkvbyg7PBLvYDXu3rYqkJRDytl/tUofs+UzQ6Mt3L8vo+SoATIQ6kqcS6oUvF8IG4dYBhAE4B3VI4XQAhwG0sIGuNsLVCLm1SkhHIOSN2uoEdWjouwByoH4z9h8AnWTOV3eNziONbibUa+5NBeDyJpYvgBpQB29IgjpEdBqAUwDCZNTsA/UczIea65wNIAHqCGNVbJDnUlF/7WEjS1vZ2stmAFgP3dDyRbfYN+k6vwa6dnn22uF+tlv9hXrIob3uZ3uVr62fCXaxG/aykwbSEgh52692qUOvwTNBtntZtjeSHA6Hw+FwOBwOh8N5M5Er2A6Hw+FwOBwOh8PhcN5QuCPJ4XA4HA6Hw+FwOByz4I4kh8PhcDgcDofD4XDMgjuSHA6Hw+FwOBwOh8MxC+5IcjgcDofD4XA4HA7HLLgjyeFwOBwOh8PhcDgcs+COJIfD4XA4HA6Hw+FwzMLBrIPdvKjMWwFypQUAkPIg4TkRvVUadIXMZ3j+/Dkrquno7QdXR4VsuobyynW5rhy6b2Ld5bpc903S5c8irlsadN/Eust1ua4tdfVRoiPJGIsEEAkAzhXeRsep30uQPMPsGNn8fmnRvfNdFPRpvh25FHX9y8imayivXJfryqH7JtZdrst13yRd/iziuqVB902su1yX68qtWyJEZPJWtkptkhsAF0qLbuPGjUmfZp9Vv8iqayivXJfryqFrDn/88QcBoMjISJvqWgLXNU9XqVQSY8zmupZSmnSlfBbdvHmTHBwcaNu2bSUe+7rZKkt1Dxw4QIIg0J49e2yqaypc982tu1yX69pSV9/2xs+RfPr0Ke7cuSNuf//73/H06VN7J4vD4eihZcuWEAQBnp6e9k4KR0I2bNiAgIAAdOnSxd5JKVXMmjUL/fr1s5neo0eP0LVrV+Tn5+Pu3bs207UnaWlpCA8PB2MMly5dsndyOH8hvv76azRo0AANGjTA7NmzxW3ZsmVISUmBSqWydxI5NuDzzz+Hs7Mz/vvf/9o7KRZh1hxJc4mPjxf/3717N6ZNmwbGGP7v//4PY8eORe3atcEYM3IGy+nevTsOHDhQbP+hQ4fg6+sri6a9UKlUWLhwIZKTkzFv3jzk5eXB3d3d3sniSMTFixfRpEkTCIKA3bt3o3v37pKct6CgAEqlEq1atYKrqyu6desGQF13KlWqBGdnZwiCbfua0tPTAQDNmze3qa7cJCYmonbt2iUeFxYWhrlz56JWrVqS6CqVSsTExCA5ORknT57EiRMnsGLFCgwaNEiS85vC0aNHMXToUADA3r17baZrD/7880988skn2LVrF3JycnS+q1WrFrp3745Zs2bBwUHWRy8AYOnSpYiOjsbmzZtl19KyYsUK3Lx5Ey4uLujbt6/NdO3F9evXUb9+fTRs2BDHjh1DmTLSD9c8duwYLl68iEuXLmHHjh0YPXo0lEolRo8ejfr160uuZw5lypTBunXr0LNnT7um46/I+fPn8emnn0L94gf4448/xO+ICB999BEAQKFQYO3atRg4cKBd0vlXJy4uDgUFBfDy8sLLly9x7do1AMDw4cPh6upq17S9evUKPj4+AABHR0fk5ubaNT2WIksrMTk5GUuWLEFwcLC4TZ8+HYIggDGG1atXIzg4WJbeFqVSiYCAAOzbtw8AUK1aNTx79gxpaWlIS0tD27ZtJde0JyqVCv/+97/x6aefgjGG8ePH4+233y7WiPmrkZOTgy1btmDSpEkIDAyEu7s73N3dsXTpUqxatQqTJk3CO++880b3ABcUFGDo0KFo27atWHcmT56MoUOHYtq0aVaX8bfffgt3d3dcvHgRcXFxmDx5MiZPnoxatWrB3d0do0ePRmpqqkS5MR2FQoGKFSvaXFdOjh07pvM5LCwMCQkJICIkJCRg+fLlANQdbrVr18aKFSus0svMzMSmTZsQFBSEgQMHYvLkydi/fz+ysrIQERFh1bnN4erVqxgwYAAAYNKkSTbTtYSzZ89a9fv79++jWbNm2LJli966mZiYiPnz52PEiBFQKpVWaZXEs2fP8NFHH+Hdd99F7969ZdXSkpCQgJUrVwIA/va3v6F69eo20bUXycnJaNeuHQBg2bJlsjiRT548Qbdu3fDJJ59gx44dAIDly5djzZo1aNSoEXbv3i25pqls2bIFGRkZ4jWwBUqlEikpKfj5558RHh4OZ2fnv+yb70qVKqFy5crw9/dHt27d8N1336Fnz57w9/dHmTJl4O/vD4VCAZVKhenTp+PZs2eSp+H27ds4duwYMjIyxH3Hjh1Do0aNoFAoxK3oMdYwc+ZMZGVl6f1uz549eP/99yVrdxw5cgTt27dH27Zt0ahRI3To0AHjxo3DuHHjUK9ePbz//vtYtGiRXdrLp06dwttvvy1+VigU8PDwsHk6JMGU8a+mjsnNz8+nsWPHUv369UkQBKObm5sb5eXlmTQm19SxwHl5eTRz5kxycHCgypUr09SpU+nRo0cm/dZU3SdPntBnn31G3bt3p4YNG5IgCMQYI0EQ6LPPPqPY2Fh69eqVSZpSzEuJi4sjhUJBI0eOpOTkZJoxYwYJgkCxsbFm57Uk3RcvXlD16tWJMUYLFy6k3Nxck9Npju7p06cpICCAGGMlbv3795dM11xSUlKobt26JAgC+fj4SK7brVs3UigUpFAo6JNPPqGpU6fS1KlTycXFhRQKBQ0cOFBvGZiqO2LECGKM0Ycffki9e/emdu3aFbu+X375pcnXw5q6q0WhUFCLFi3M+o0UupZgji4AcUtISDB4zuXLl4vHhYWF6T3WFN2hQ4carTdPnz6VNb9ERDk5OdSuXTsSBIGqVKlCaWlpZmtaomuImTNnUkBAAC1YsIDu3LlDGRkZlJGRQQcOHKAaNWpQp06drNJdvXo1CYJAfn5+VKdOHapTpw6NGjWK6tSpQzVr1tR5/k2cOFHW/IaHhxNjjBYtWmTS8VI8i9q1a0cAyMXFhf7880+TfiOXbbaF7urVq4kxRj4+PpSdnS2L7qNHj8jFxYW6d+9Ox44dE7dhw4aJ7Y47d+5IrmuM3NxcmjJlClWoUIEYY/Ty5UtZdFUqFd29e5du3LhBy5YtoyFDhlDLli3FZ6JCoSAnJye6cuWKybrGUCqV9ODBAxoxYgQpFAqKjIykBw8e0IMHD4gxpqPbsGFDve08qWyV1j6PHTuWHBwcSBAEioqKMni8ubrZ2dl09epVqlq1KgmCQB988AFdvXqVevbsSYIgkIODg84mCAK1atXKKt2CggI6ePAgeXp60pAhQ/QeExwcTIIg0PLlyw2m3VTdxMREql69eom+iCAIVL58eVq+fDllZGRYrWsKJ06cIDc3t2LPZT8/P4t1L168SACodm3D6VGpVHTr1i1KSkqiDRs2UHp6utF06tPVt0nmSGZkZNCwYcNKLLABAwZQREQE3b9/3+ILZoh58+aJN/7ly5dN+o05uk+ePCFPT08SBIEUCkWxv9r/AwICqH///rR//36jmlI8vLXa165do0uXLpGLiwsdPnzY6IPNEuP+4sULCg4OprJly9Ls2bMJAC1btszkdJqqm52dreNEVq1alcaPH09bt26liIgI8vX1JT8/P+rfvz9t3bq1xIpgaX4N8ejRI4qIiKC6deuSp6enzsNFKt38/HyKjo4mBwcHKl++PD158kTn+9zcXHrx4gWtXbvWKkcyOTmZ4uPjSalUEpHa0MfHx1O/fv3E6x8TE2PqpbHayK5bt44UCgWdP3/e5N9IoWsppuoWdg6NOZFaEhISdJxJS3S15VehQgWKi4uj+Ph4io+Pp8jISGKM0e3bt2XLr5aTJ0+Kdr+khoGUuvr4/fffdex0YXut/X/Hjh1W6a5evZqmT5+u99rm5ORQ3759xevRtWtXWfPLGKPJkyebfLwUzyLGGAGgNWvWmPybv7IjWb16dfLx8aHr16/LqpuSkkI5OTk6+woKCqhVq1bEGLNpUKPnz59Tx44dRfsydOhQvS8ELNF9+vQpxcTEULNmzahZs2YUEhJC7u7uxepp4c3b29ssXWMsXryYHB0d9W4KhaLY5+nTp0uiWxKTJ0+W3JFcsWKFjpNY1GnU50g6ODjQsWPHLNYtKCgQ7V+9evX0HqN1JAVBMHJFTNMNDQ0Vz9WzZ0/avn07ZWRkUGpqKm3fvp26du1KLVq00PFPxo8fb7VuSRw/fpzc3d3FOtS5c2eqV68eMcbIxcXFYt26desSY4wcHBzIz8/P4Obu7k6enp7EGKMBAwaYnV99myQTNVQqFT766COsW7euxGPXrFkDFxcXKWR1WLJkCWbNmgUA8PHxQYMGDSTX8PDwQJMmTRAbGws/Pz/8+OOPqFatGvbs2YPr16/j6tWrePDgAZ4+fYpt27Zh+/btSE9Ph5ubm+RpAYDHjx8DACZMmIDatWvjn//8J/Ly8tCxY0fJtT788EO4u7vjwYMH8PDwwB9//IGPP/4YXbt2RdWqVSXTmTp1KpKTk1G5cmVs3rwZTZo0Ecex9+vXD5mZmQAAV1dXWebvnT17FmfPnoWXlxf27duHq1evit8VFBQgMzMTKSkpkusW5sGDB+K9/MsvvxQbx+/o6AgfHx+8//77SEpKQrVq1SzSqVChAipUqCB+ZoxBoVDgyJEjqFq1Kg4ePIiaNWtanhEzWbhwIQAgJCTEZppaVCoVrly5gsTERMyZMwd5eXkAgHLlymHhwoVo1qyZJPO5TZn7WKtWLRARGGMWD12rU6cOWrdujTlz5qBs2bIA/nf/AoCXl5dF5zWVU6dOiXNuFy1ahMjISKPHjxkzBpmZmfjss890hvtIxc6dO0s8xtohoAMGDICTk5Pe+Y/Ozs4YOXKkOLw4NjYWp0+fRsuWLa3S1MeGDRvQvn17zJkzR/JzG+LEiRMgIsybN8+mQ6ftyf3799G6dWvUrVtXVh1vb+9i+7S2GgBu3rwpq35h3N3dxeeRj48Ppk2bJtl83/j4eAwbNkwcQskYg7+/PwC17Ro8eDBq1KiBzp07o0qVKgCAuXPnSqK9Zs0aTJgwweYxAUxB275au3Yt5s2bZ3V7Mi0tDdHR0Rb9dtWqVZIMZbbFvO1Zs2ahcuXKmDp1Knx8fHQC+PXt2xd9+/YFEWHXrl0YOXIkUlJSsGnTJkyYMAGVKlWSLV3dunUTh/b269cPa9euxZw5c8S5m5aijUmjUqlMDij6888/Y+/evfjnP/9plbYkFmDChAkmOZEAMHr0aKxcuRJOTk5SSIv8+eefUCqVqFy5Ms6fPy/pubW4ublh3759ePHiBcqUKSM2xoYPHy4ek5WVhWvXrmHNmjX4/vvv0aBBA5w5cwZvvVXimp5mkZSUhJCQEAQFBWH+/PlYsmQJ/vOf/8gSZGfgwIHYuXMnfvnlF3EMd+fOnXHgwAF4enpi0KBBeP78OXbv3m11J8GSJUsAACtXrtTbyNLOLVIqlXBwcJB0XsqpU6fQqVMn0YkoirZxrw85gg04OjoWc+SuXLmCPXv24OzZszh37hzS09ORn58viZ5KpcK7776LtLQ0XLlyxabzFF+8eIEXL16IDrQtyMnJwatXr3DixAn8+9//RvXq1TFgwAB89dVX4jEpKSkIDQ3F5s2b0bVrV4t0Ro8eDQDiPEhTSUhIMClAjz4uXLgABwcHODo6ivvu3buHrVu3AgDKly9v0XlNJTY2Vgyc5ODgoLexmZOTg9WrV+Obb77Bs2fPkJubi5iYGPF3UqZlzpw5cHZ2Rnx8PAICAhAeHo6YmBgQEby9vfH999avx1VSA69NmzZo1KgR4uLikJWVhWXLlqFZs2Y6ZWQtV69exdy5c3Hs2DGbBPTREhYWBgDo37+/6OBIzdmzZ/H8+XOd+qmPwna6UaNGWLBggaTXGAB++OEHVKlSxaQOCjnYt28fzpw5AxcXFwwePNhmupmZmbh9+zYAIDo62uJOTH20atUKN27cECNXOjg4oFmzZsWO08a56NKlC0aMGCGJ9rZt2yQ5jxwMGTIE06dPR0pKinoYoZVs2LABL1++NPj9kSNHirVzOnToAACIioqySDM3Nxeff/45AOBf//oX6tSpo/e4pUuXon379hZpFKVx48b49ttvjR7DGEOvXr0AqJ3Lly9fYu7cuVbHKDBG69atceDAAWzevBlhYWFQqVRYvXo1HBwcxHnQtiIzMxMzZsx4PRxJQz1iI0eOBABs3LhR9MDXr1+PgoICLF26VLKJpffv30dcXBwAoH379pI7bYVxc3Mz2mBwc3NDSEgI/P39sXbtWty6dUvyibwPHz5Ey5Yt8fTpU1y+fBkAcO7cOTDGzG6smkJKSgqaNm2Kxo0bi/vq1KkDd3d3/Pbbb/jhhx+Ql5eHhw8fWv0Gq6CgAIGBgTq9Xqmpqdi/fz8OHz6MjRs3ivs9PDwQGhqKgQMHok2bNlbfTy9fvjToRBojKCjI5I4UU1i1ahUA9YO6oKAA//3vf9G6dWswxvDs2TOoVCoIggB3d3fMnz9fEs27d+9i2rRpSEtLAwCbB7tZvXo1njx5gokTJ8py/vPnz6NWrVpi7/6GDRuwbNky/P7774iJiTHa+XT06FFERUVZ7EhaSq1atRAWFoaePXti165dZv226Fvsu3fv6kR4XLVqFRo1aiTWaakb/9q3Cl999ZX4HCjMtWvXMGnSJBw+fFhnf1ZWFnJzcyXpaMzOzsbEiROxfv16MMYQHx8v9u5v2LBB7LTw8vKCn5+f1XqmsG7dOtFGxsTEYMOGDZI5Oc+fP0fPnj2Rk5Mja496UR4+fIjMzEy0bNkS5cqVk/Tcubm56NatG169eoVLly5BpVKVODKgsCN57tw5pKamYsOGDZKl6eXLlxg5ciQ2bdokRlyUm1evXuHly5dISEjAqVOnsGjRIuTl5WHMmDGSjggqiVGjRiE+Ph4KhUKWkU8BAQEICAgw+P2FCxdw5swZAEBgYKAknSWpqalITU1FQUGBzv7WrVsDAH777bdigWZcXV1tdt1dXV3Rr18/SZybly9fYt26dTp51f4/YcIEjBw5EjVq1Cj2O5VKhaysLIvfhi5cuBDffPMNAHW5GbJ52tEztiQhIQHHjx8XPz948ABKpRLOzs6y6G3YsAGZmZnw9fWFQqHA559/jmfPnsHLy8tqh84YDg4OqFGjBhITE6U/t+RnhLrndeLEiXj//fcBAH369MHu3buxbNkyAGrHUqVSYd26dVY3YFJTU9G9e3dcv34dALB//37rEi8hjDHJh0ocOHAAo0aNQlJSEkJDQ+Hn54eUlBScOHECwP96hqWmaE/Y8OHDMWjQICxbtkx0vqQaBtmlSxcIgoDExERs27YNixcvRmpqKlxcXFCuXDnxmubl5WHnzp3YuXMnXFxc8PHHH+OLL76w+J7y8/ODs7OzWREVPT09sWTJEkmHL2dnZwOBeOr4AAAdIklEQVRQL4URHh6OHTt2iI2j8uXLIzIyEj169ECjRo0k0Xv48CEaNmwoOpHjxo2T5Lym8uzZMyxfvhxEBCcnJ+Tn5+P06dNiL+ivv/6q04lhLtnZ2Zg7d67Y4fHpp59i/fr1+PLLL9GvXz+jHRA5OTn4/fff8ejRI4v1tVjS09q+fXuMHj0aiYmJFi8JsmnTJkRFRYnDWgF1g1CLo6MjoqOj0atXL0nqcHZ2Nk6fPg0igouLSzEbqFQq8cMPP+A///kPiEiMVpeXl4fs7GzcvHkTwcHBVqfj/v37OHHiBHJzc/HTTz/pNPxcXFwQFBRktYa5yLks05AhQ3D37l0cPXpUNo2iaIez5uXlYc2aNZJP4wgNDUVsbKz4uaCgoMRnatFjfvzxR8kcyfz8fMybNw9paWk261hasWIF5syZg+Tk5GLfJScny9roNURYWJhkSxSZyq1bt/CPf/wDBQUFqFmzpviGy1quXr2Kx48fQxAECIKA7t27o27dupgxYwZu3bqFbt264datWzr31OLFi8Uljf5KODk5wdPTs1gd8vf3x9dff230t5bW7SdPnogjPv71r39h9uzZFp1HKpRKJe7du4fTp08jPz8fU6dOFaPEuru7Y8mSJZLWp8zMTDx8+FD8PGHCBFSsWBFTp05FTk4OFixYAAB45513rNKZOXOmOD2oMIsXL0bZsmVRpkwZNG/eXPLRoACsC7aTnp5OERERxaKx/vTTTwYnb0ZFRRmdcAszJrMOHTpUnAx89OhRWr16tcEJw506dTIaSc4cXVMYPnw4KRQK8vf3NxjF1dQABykpKdS0aVNydHQkQRBo0qRJlJ+fL34/fvx4UigUNGHCBJPSZiivxibex8TEkK+vL3Xq1EmcJFyvXj3x85QpUyTRLRrFKjw8nG7evFniuc+fPy/+Ror86mPt2rXiJH9PT086evSo0eMt1Y2KihJ1Ro4caXLkYUt109LSaMWKFVSjRg2daw9NBLAtW7ZYpWuMBQsWiHmdP38+Xbp0iQRBICcnJ6pdu7b43cCBAy3WDQ8Pp5YtW1K/fv3MjlZavXp16tixo1lBjfQcRwAsDjiDIkF6zL3OaWlpdPHiRXFLSEgQ/1+2bBl17tyZAIjlPmXKFIvzW1BQQOPHjycXFxcaOnRosXO8ePGCGGPk6OhIffv21fnu5s2b5Ovrq2PbLMkvEdHnn38uBuho3rw5hYSEiNGVTbEnpuouX77caHA5R0dHGjFihLiVK1dO/O6zzz4rlldL8qtUKikoKIgYY3Tq1Clx//nz56lKlSric0MQBPLy8tIbiM3SYDtr1qwhACQIAl2+fJkaN26sE6HYy8vLqqie165do2nTptG1a9fo2rVrlJKSYjQ9mzdvpqCgINFuBAYG6s2vpbb5/v37pFAoKCgoyOhxhjBXNz4+Xscmu7m5ic+D/Px8+uKLL4gxRgEBAfTixQvJdPXRtWtXYoxRr169TP6NVM/ezZs3i2V648YNi3XNoXbt2sWC7Rw8eNBq3R07dhi0FREREXT+/HmdgF2LFi0SjzEUVdSc/Kanp1Pv3r2pTZs2Om1lDw8P+u6770y8Oqbruri4kKenJx05ckTcp1Qq6cqVK3T58mU6fvw4BQcHU926dcnFxUXMqzYqsb7AO5aWb+EgN4a2EydOWJVfInXw0c6dO4vn3LNnD/3+++/0+++/U0xMTDHNefPmGU23VP7JjRs3dHQDAwONti/16erbrHIkb9y4UawyVKhQwWhGsrKyaOzYsZLcIPfu3aNy5cqJjmR+fj5dunTJYOSpNm3aGIzKJKUjuX//frESGIvcasrDOz8/n+bPny9er7CwMIqPj6e7d+/S/Pnzafr06eJ3ISEhNH/+fPHBPnz4cLPyWpJxz87Opvz8fMrKyqKsrCxSKpW0ZMkSkyqCqbpBQUHUvHlzOnLkCCUnJ5d4zsL89ttvsjmSeXl5FBoaKj7MjEVQs0ZXG11Su82fP19vY1NqXSKi1NRUWrlyJUVERFCHDh1EY6NQKGjAgAElLt9gSR0KDw8nhUJBvr6+lJqaSh06dCBBECg6OpqeP38uiSOpUqkoNzfX7OVq9u7dSwqFgn799Ver8iuFI2mJrqmoVCq6ffs2ffPNN+Tv70+MMZo0aZK+dJSo++LFCxIEgby9vfV2QPTt25cUCgWFhobqPMBycnKoQYMGJAiCZI5k4YaIIAiiXZw1a5Ypl8Uk3YEDB5oUXr7oVrlyZYqPj7dYtzAJCQlix6z2Hl+0aBExxqh69eq0YsUKWrBggdg4y8rKKnYOSxzJhw8fUrly5QgAVatWTceB1G5OTk507949g+eQytEgIlq/fr3YENU6kQ8ePJBU94svviCFQlGiQ2EIc3WVSiV16NCBunTpQtu3by/mLObn51NYWBgxxujcuXOS6eqDMUbdu3c3KVK6lLoPHz4kHx8fUigU1KxZM5Oeh9bayCdPnuhEh/Xw8KC1a9dKotunT58S7YOTkxMtWrSIDh48KC7RMXDgQFKpVJLlNz09nbp3716srTxx4kSTO1xN0RUEgTw9PWnAgAF08OBBGjBgAPXo0aOYs1jUXvv4+FC9evWsfuFUmHPnztHkyZNpzZo1tGbNGkpKSqLU1FQ6fPgwubm5kSColznJzMy0OL8XLlygZs2aEWOMypUrR4cPH9aJunzv3r1iHfZ79uwxmm4pnvm3b9/WicjPGKPZs2ebratvs8qRzMvLo1mzZpnlSBIRbd26VRJHcty4ceTk5EQTJkzQCUGdlZVFiYmJVLly5WLOZMWKFU2+YJY2zubOnSs+zAzdkESmPbyzs7OpWbNmesPVGwqL3aJFC1q8eLFBgy/lw3vIkCFUrlw5o72h5ujm5+cbNJYlkZWVJZsj2aVLF51rbYqhtUT34MGDpFAoqEqVKjR//nxSKBS0c+dOk9NpqW5RcnJy6MyZMzoOZdWqVY0aPEvepHTu3JkUCgWNHj2aiIicnJxIEATaunUrnTp1Srze06ZNk0zXFJ49e0a1a9cutragJbowspRHSWiXDrFE1xLmzZtn9pv9whTuPT979myxcwiCQO7u7jqdEleuXBHXmyzca22OblEKL/fRtGlTcY1HhUJBffr0KekymKxb2JEcP368uMyKdqtSpYrehqKUy3/s3buXBEGgJk2aUHp6OuXm5pIgCBQUFESxsbGkVCrpwIEDJAgC1a9fX2+HiiWO5JgxY4o5jp6enrR06VLy9fUlAFS5cmWDvzeWV3NsVUZGBs2YMUNcX1cQBKpYsSIlJSVJrtujRw/xuf7nn3/Sjz/+SB988AGFhISYNHJDymevFu1bBkvuKVN1r1y5QgDMWupECl0iIh8fHwJA9evX19sJYo6uqbRr105nuQ9TnEhTdbUOk7u7O1WsWJEmTpxIa9eupZUrV1LFihV1Or0KbyNGjKCCggLJ89u1a9diHW6mLuFjqiNpaDPmSBobOSLHM3Dp0qWi9siRI/W2Q03RbdGiBTHGaNSoUXo70aZNm1bsjWRJbQMp8jtmzJhiusY6+Qzp6tusciSJ1N535cqVTXYkMzMzaeTIkVY7ko8ePRJfzU+dOrXY9xkZGTRq1KhijqShtbWkujFjY2PFXv0ePXoYPdbUh/e5c+eof//+1L9/f+rXr5/4/6lTp+jDDz8Uh0Bqh/+UhFQPs5ycHKpevbpJw1qt1c3Pz6dbt24ZPUZOR7KwcdfX2NWHubrnzp2jgIAAUigU4iLL2uHR5iBlY0WpVNLJkydFw+Pk5GSwgWZuHXr48KHoKPbo0YOUSqXYEKxRo4b4XWRkZLF11KzRLYnc3Fxq27YtMcaMrkdrqm7hdSTNZfny5cUeMnI6koWH0hXFXEdy/fr1xc6hbTBMnz6dTp48Sd26dSNfX18SBIGOHj1q1RDioqxYsYJWrlwpNj61juTgwYNL/K2puoUdyb59+xZr6GqdOjkdSaVSSYGBgWJDrF69eiQIAvn7+1NQUBAFBgaSIAjUu3dvg6MKLHEkL126pONEli1blhITEyk5OZk8PDwIgNGOGGN5NcdWTZw4UefaVqxYkR4/fiyLrvYap6en623wW/pmwRpHMiMjgxhjBtdUlEK3V69eFBISYnCajly62meCu7s7nTx50mpdU4iNjaWKFSvqOJJS6mrvoaFDh+p1VrTraOuzG8OHD6cxY8bQmDFjJMtveno6hYeH60wLq1GjhtG1yM3RRaFpE4wxqlmzJs2cOZNmzpxJM2bM0DlWuxZiSW3nknSTkpLo6NGjZr09z8nJER15Ly+vYut3m5pfxhj16dNH77MsLS1NvA4BAQHieo6urq509epVi/NrCkUdyXLlypU46s9mjiQRUefOncUb3c3NzegCucOGDROPbdCggUkJ16e7ceNGcnBwoGrVquldAHrQoEHFnEgHBwd6/vy53nRJUVBPnjwhf39/sfFvaFiNFikWgR44cCApFAqTHEgtUj3MMjMziTFGGzdulF33yJEjejseCqOdfyWlbnZ2NoWFhYkOTosWLUwysJboat9GdurUSXzAaJ0pc5C6sVJQUEAzZ84UDdCHH35olq4hUlJSqGrVqmIeQ0JCir1l79q1q9E3C5bolsRXX31FTk5ONGvWLIM9wOboJiQkiI3twnMdS0L7u6K/+as4kh4eHsVsc9++fYs1kvz9/WnRokUGhx5LkV/tG8ouXbqU6GSYo5uUlKSTl8uXL4vOZFpaGn3yySd6G4StWrUy+HbFkvympKTQ9OnTqWbNmmL5NW/enMLDw01qgEvhSG7bto3mz59P3t7eBIDc3NxKnNNora1KTk7WGY0zadIkk8rXUl2tbbp48SIpFAry8fGhUaNGkZubGykUCoqOjpY1v/pYvnw5McaoXbt2suk6OTmZ1SiXSvf06dOkUCho2bJlkuiawrRp03ScyNOnT0uqW9j+ac/96tUrOnXqFJ06dYpcXV0NOpKF7YdU+dVSNL5I7969S2zrmKL74sULnc3YfRQcHEyCINCuXbus0nV1dSVBUE+x0A6pLckWEak7H7XXWF87x5T8enp66p3He/v2bbGToEmTJpSSkkLnzp0THe0dO3ZYnF9TGDp0qI4jGRkZWeJvTHUkJV99NScnB3v37hWX+9CSlZWF4cOH60RPk2KJgadPn+LAgQPi5/z8fHz44Yc6C546ODjAz88PW7du1bu4r1R88MEHePz4MYgIM2fOROXKlWXT0rJlyxbZNQxx6dIlm2llZmaqez4MoFQqMXbsWEk1k5KSEBQUhJ9//hkAEBISgsOHD1u9VqY+cnJyxLIUNNHjtNgylL8+GGM611alUklyXm9vb53zXrx4Uef7mjVrYteuXTZdiiQjIwPR0dH47rvvMG3atBKXGjCFWrVqicvy1K5du8Tw24mJiVixYoW4hqRU0RFNiUZs7RqOw4cPF6MJZ2VloX379hgwYAC++eYb1KtXDzExMTrHv/XWWzh58iTGjRtn1TIYt27dMpr2b7/9Fj169ECLFi0kXerDz88Pq1evFtfTa9y4MYYMGYKlS5eiS5cuBpfniYuLK7b0iTV4e3tjxowZuH79Ou7du4f27dvj5MmT2LRpE1q1aiWZjjH69euHyZMn49WrV3B0dERsbKxsz9vs7Gx89NFH4lIFvr6+OHHiBL788ktZl3LRRl5u3bo1Ro8ejQsXLiA4OBiOjo5wcnIS1zi0FdnZ2aJtsXSReWOoVCoMGzYMeXl5ki3XZirXr19Hx44d0bt3b71LCElNbm4ulixZgi+//BIFBQUoKChAtWrVJI9O++mnn4r/t2nTBhUrVkS1atXQpk0btGnTRrTTLi4u8PT0hKenZ7HlnORoe1WpUkXn848//mhWBHtD+Pj46Gy2uI+0bbS0tDRs27YNXbp0Qc2aNfHee+9hwoQJuHjxos5Sb9nZ2cjOztZZUqakNSgN8c4774jXMj8/H4mJifj888/x3nvvITk5GX//+99x5MgReHt769xbUq+fXJTCS9QpFAr0799fupOb4m2W5AEXnvOoz5vfv3+/znBW7bZ9+3aTPGB9uufPn6fAwEBycHCg/v3704ULF+jChQu0ZMkSnV6VLl26FHt9bqrnba7H7+npSYIgmOTpE0nzRhIAtWjRwqx0Gsqrub2izZs3t9kbyZ9++okYY3p7sx4/fkyhoaHi0EspdBMSEmjjxo06vd3G5rvqwxzdGzduiDqhoaHifoVCQXPnzpVN1xRUKhVdvXpV7MmaPn26WbrGyM/Ppzt37ohv1gu/kaxfv75J6ZOi7hKph9pWr16devbsKdmwHi2F30pCM19S39vJwsNgtcdZo1uYDh06GIw0nJ+fTytXrqTKlSsTY4x8fX0t1j137hw1adKkxHkxo0aNMmnOkym62nN27NiRLly4QHFxcXTz5k3as2cP7dmzh3x9fcnV1bXEOSGW5JeI6LPPPqOaNWsazHO3bt0oJSWFrl+/Lu7z8/PTO2Tb2vv5+fPnNG7cOJOPJ7LsWXTz5k1ycHDQuV/Lly9P0dHRJr+9stRWbd++XbQXI0aMMOltgxS6N27coPLlyxezVa6urnT48GHZdIuSnZ1NR44coeDgYGKMUceOHY3WJUt1Fy5cSIwxCg4ONit91upmZmaK8+fNLVtjusZ48OCBzpvIpk2bljidxhLdO3fuGA24U6FCBYqOjtYZRpyRkUGHDh2iunXrUs+ePYvZDSmegbVr1y4WqLKkocxSPXuJiBITE8Wo1ta+kczMzKQLFy7QqFGjxLeTRbeGDRvS4MGDafDgwXqP0Tdly9RnUbNmzWjWrFnFgts0bdpU59iUlBTxOy8vL51YL+bqlkThdBiLOVGSrr5NEkdSpVLpDYM+aNAgGjRoELm7u+vsHzFiBF2+fNniyaxazp8/T97e3nqHsGqXLTAlCIy5uobQDucydaKyFI6kIAjUsmVLs9Ip1cOsQYMGxBjTO5Zcat3U1FQKCgqicuXKUUREBG3ZsoW2bNlCEydOFEM6KxQKmjhxoiS6HTt21HEizWmAWqL74sULCgwMJIVCQR07diSVSkXZ2dmkUChMHopniW5hCgoKxCAhhbl8+bJogIwN7bWmDqlUKp05kpGRkSZHWZWi7qanp1PTpk3Jw8PD6HxMa3SLOpPQRHIt6jwW/k7K/DZu3JhcXFx0gje9evWKrl69Sj169NB50OgLqmGObnp6Ok2ZMkXv3EClUklKpdLkaMSm6H777bdUsWJFg4HIwsLCzK7D5l7n+/fv68QLEAT1Eh8//fSTODcxKyuLBgwYIH5vztIUppKZmUkdOnQw+T4msvxZdODAAeratSs5OjrSlClT6P79+yZrEllmq6Kjo8WhpAqFwmDjS2pdLSkpKeTr6ytGm96+fTs9e/ZMdl0itZ2MjIwkb29vHZtcUvA3S3Rzc3MpIiKCGGMmt6Ok0CVSD8NTKBRUvnx5ScvXGCNGjNBxJA11mEqh++jRI2rZsmUx+1i2bFmjU6JSU1Ot0jXE1atXqUKFCjrzfo0Nv5dKtzA7d+4U7+mYmBjJdPPy8ujMmTO0cOFCatasmdEhw9oymDdvnsX+ib4lRbRTd4rOUU9JSSEnJydijJGHh4fBZ5S11zk1NVUnPS1atLC4E1ff5mDZe0xdBEFAp06dEBoairi4OPEVbeHhpVrat2+PTp06oUGDBlbrhoSE4NChQ3jvvfd09u/ZswcNGzZEhQoVrNYwB8YY3n33XQwbNsymuoMGDbKpnhb1fQZ4enrKrlWmTBlERERg6tSp2LBhQ7EFpgVBwLhx4/DNN99YrRUXF6czdMTZ2VlnMXM58PHxwfbt2/Hee+/h+PHjOHPmDE6ePAkvLy/JF/k2xKFDh9ClSxcAwC+//IKlS5fi8ePHOHHiBACgbNmyWLFihSxDewVBgJOTE/Lz8yU/tymsX78e58+fx+TJk2Vb3LtWrVogIvTs2RO7d+8GAIwePbrYcWFhYWjfvj1GjRolqf7ixYvRsmVL9OnTRxyCd+/ePdy9e1c8pkKFCvjkk08QFBRklZaHhwfmzZuHefPmWXUeU4mMjMSgQYOwY8cObNiwASdPntT5/qOPPpK9DlepUgW3bt3CoUOHEBgYiIULF2LGjBlQKBTiMa6urmjatCm2bdsmWzrc3Nxw/PhxtG/fHrGxseJQLTkIDQ1FaGiobOfXx6tXr5CTkwNXV1ecOXNG1vzpw9vbG0+ePLGJVkFBAVQqFQ4cOICYmBi8evUK+/fvB6CesjN79mxERUUVG/poLXl5eZg3bx42bdqEuXPnomzZspKe3xA5OTn4+uuvsXXrVgDAyJEjbVa+33//vc6UkokTJ8qmFRAQgCNHjmDJkiXIzc0FALRs2RINGzY02p4qU6aM1doZGRmIiIgQPxMR4uLi8PLlSzH/zs7OstxXJcEYw9tvvy22Q6TAwcEBzZs3R/PmzTFmzBikpaUZPd7R0dGq67x+/XoolUpMmjQJwcHBGDx4MN555x00b9682LHe3t5IS0vDxo0b8Y9//EO2Z9TDhw91Pk+aNEnSspWshtaoUQP79u0DoJ7PFhUVJTb2z5w5A29vb1SuXBnu7u5SSQIAmjZtqjPW2V60a9cODRs2FK+BLXWjoqLg5uYm7ZhnE5gwYQIiIiIQExODgQMHyq736aef6swvkIMTJ06gY8eO4mdHR0ccP35cVk0tTZs2hVKpxL59+xAeHo5nz54Vm2ssJx06dMDRo0fRoUMHbN++HY0bNwYA7Nu3z+YPFFsTGhqK06dPo0WLFrJr7dq1S3YNfbRo0QJKpRK//PILLl68iK+++grBwcEYP348hg8fDkdHRx2n56+Gi4sLBg0aZLeONUDdAOvevTsAFOvs0hIVFYWoqChZ0yHVPObXkUWLFmHRokX2ToasXLlyRZxr3Lp1a7zzzjuoVq0aunTpgr1798qun5SUhC+++AJ79uxB165dZdcD1B2ZWq0pU6Zg5syZNrFHt27dQrdu3XT2DR8+XBKnzRjOzs6YNGmSrBr6cHNzwwcffIDly5cjLi4OBQUFOg70kiVL8H//938276Dp1auX7HbL0dER5cqVk1VD+/wZMWKEScc7OzubfKy1MMawY8cO8RklFbLcKe7u7vj+++/x/fffy3H6146srCwkJSWhb9++Nnt7pOXIkSM21StMSEgIAPWkZFs4krYgODgYFStWxKNHj8TPTZs2tZm+IAjo1q1bsQebLXB0dES7du1QUFBgc217U6NGDTFwx5uMo6MjWrdujdatW2PChAn2Tg6Hw9FDgwYN7GqHq1WrZjf9SpUqYfLkyTbr1Nq4cSPu3Lkjfh47dixmzpxpE217IAgCevXqhc6dO+O3334DEWHKlCmYOHEi/Pz8bB4wimM7nJ2d0bNnT8nPa9suhzcUNzc3VKpUCZGRkfZOik2pVq0aqlevjqlTp9o7KZLx1ltvYdWqVeIbg1mzZtk5RRwOh8PhvNl07tzZLlMbPv74Yxw+fFiMGt6hQwebvxCwBx4eHqLT+Ntvv9k5NRw5kTvyPXckJcJWwx9fJ5ydnXH79m17J0NyQkND38h8cTgcDofD+R9eXl44f/68vZPB4ciGt7e3rCMMmDZgikkHM5YOwPgiaNZTlYjeKiW6+jSfAcgE8Jzrct03QPdNrbtcl+u+Sbqvk83gulxXDt03te5yXa5rM119mPtGMpGImliYIGsoNbpE9BZj7ALX5bpvgi5KUd3lulz3TdItbbaK677ZuihFdZfrcl1bIpR8CIfD4XA4HA6Hw+FwOP+DO5IcDofD4XA4HA6HwzELcx3J1bKkgutyXa77puqWprxyXa7Ldbku1309dUtTXrku17UZZgXb4XA4HA6Hw+FwOBwOhw9t5XA4HA6Hw+FwOByOWXBHksPhcDgcDofD4XA4ZsEdSQ6Hw+FwOBwOh8PhmAV3JDkcDofD4XA4HA6HYxbckeRwOBwOh8PhcDgcjln8P5xfFMDHvkg9AAAAAElFTkSuQmCC\n",
            "text/plain": "<Figure size 1152x432 with 30 Axes>"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Upload MNIST dataset to default datastore \nA [datastore](https://docs.microsoft.com/azure/machine-learning/service/how-to-access-data) is a place where data can be stored that is then made accessible to a Run either by means of mounting or copying the data to the compute target. A datastore can either be backed by an Azure Blob Storage or and Azure File Share (ADLS will be supported in the future). For simple data handling, each workspace provides a default datastore that can be used, in case the data is not already in Blob Storage or File Share."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "ds = ws.get_default_datastore()",
      "execution_count": 12,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "In this next step, we will upload the training and test set into the workspace's default datastore, which we will then later be mount on an `AmlCompute` cluster for training."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "ds.upload(src_dir='./data/mnist', target_path='mnist', overwrite=True, show_progress=True)",
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 13,
          "data": {
            "text/plain": "$AZUREML_DATAREFERENCE_3a2abea01f874563a6dceebdde6c57e4"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Create a remote compute target\nYou will need to create a [compute target](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture#compute-target) to execute your training script on. In this tutorial, you create an `AmlCompute` cluster as your training compute resource. This code creates a cluster for you if it does not already exist in your workspace."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "If we could not find the cluster with the given name in the previous cell, then we will create a new cluster here. We will create an `AmlCompute` cluster of `STANDARD_NC6` GPU VMs. This process is broken down into 3 steps:\n1. create the configuration (this step is local and only takes a second)\n2. create the cluster (this step will take about **20 seconds**)\n3. provision the VMs to bring the cluster to the initial size (of 1 in this case). This step will take about **3-5 minutes** and is providing only sparse output in the process. Please make sure to wait until the call returns before moving to the next cell"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core.compute import ComputeTarget, AmlCompute\nfrom azureml.core.compute_target import ComputeTargetException\n\n# choose a name for your cluster\ncluster_name = \"gpucluster\"\n\ntry:\n    compute_target = ComputeTarget(workspace=ws, name=cluster_name)\n    print('Found existing compute target')\nexcept ComputeTargetException:\n    print('Creating a new compute target...')\n    compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_NC6', \n                                                           max_nodes=4)\n\n    # create the cluster\n    compute_target = ComputeTarget.create(ws, cluster_name, compute_config)\n\n    # can poll for a minimum number of nodes and for a specific timeout. \n    # if no min node count is provided it uses the scale settings for the cluster\n    compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)\n\n# Use the 'status' property to get a detailed status for the current cluster. \nprint(compute_target.status.serialize())",
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Found existing compute target\n{'allocationState': 'Steady', 'allocationStateTransitionTime': '2018-12-10T16:35:24.524000+00:00', 'creationTime': '2018-12-10T16:09:58.703952+00:00', 'currentNodeCount': 0, 'errors': None, 'modifiedTime': '2018-12-10T16:11:24.446527+00:00', 'nodeStateCounts': {'idleNodeCount': 0, 'leavingNodeCount': 0, 'preemptedNodeCount': 0, 'preparingNodeCount': 0, 'runningNodeCount': 0, 'unusableNodeCount': 0}, 'provisioningState': 'Succeeded', 'provisioningStateTransitionTime': None, 'scaleSettings': {'minNodeCount': 0, 'maxNodeCount': 4, 'nodeIdleTimeBeforeScaleDown': 'PT120S'}, 'targetNodeCount': 0, 'vmPriority': 'Dedicated', 'vmSize': 'STANDARD_NC6'}\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now that you have created the compute target, let's see what the workspace's `compute_targets` property returns. You should now see one entry named 'gpucluster' of type `AmlCompute`."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "compute_targets = ws.compute_targets\nfor name, ct in compute_targets.items():\n    print(name, ct.type, ct.provisioning_state)",
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": "aks-cluster01 AKS Succeeded\naks-cluster AKS Succeeded\nmydsvm VirtualMachine Succeeded\naml-compute AmlCompute Succeeded\nautomlcl AmlCompute Succeeded\ncpucluster AmlCompute Succeeded\ngpucluster AmlCompute Succeeded\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Copy the training files into the script folder\nThe TensorFlow training script is already created for you. You can simply copy it into the script folder, together with the utility library used to load compressed data file into numpy array."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import shutil\n\n# the training logic is in the tf_mnist.py file.\nshutil.copy('./tf_mnist.py', script_folder)\n\n# the utils.py just helps loading data from the downloaded MNIST dataset into numpy arrays.\nshutil.copy('./utils.py', script_folder)",
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 16,
          "data": {
            "text/plain": "'./tf-mnist/utils.py'"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {
        "nbpresent": {
          "id": "2039d2d5-aca6-4f25-a12f-df9ae6529cae"
        }
      },
      "cell_type": "markdown",
      "source": "## Construct neural network in TensorFlow\nIn the training script `tf_mnist.py`, it creates a very simple DNN (deep neural network), with just 2 hidden layers. The input layer has 28 * 28 = 784 neurons, each representing a pixel in an image. The first hidden layer has 300 neurons, and the second hidden layer has 100 neurons. The output layer has 10 neurons, each representing a targeted label from 0 to 9.\n\n<img src=\"https://notebooks.azure.com/SergeRetkowsky/projects/TensorFlow/raw/nn.png\">"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Azure ML concepts  \nPlease note the following three things in the code below:\n1. The script accepts arguments using the argparse package. In this case there is one argument `--data_folder` which specifies the file system folder in which the script can find the MNIST data\n```\n    parser = argparse.ArgumentParser()\n    parser.add_argument('--data_folder')\n```\n2. The script is accessing the Azure ML `Run` object by executing `run = Run.get_context()`. Further down the script is using the `run` to report the training accuracy and the validation accuracy as training progresses.\n```\n    run.log('training_acc', np.float(acc_train))\n    run.log('validation_acc', np.float(acc_val))\n```\n3. When running the script on Azure ML, you can write files out to a folder `./outputs` that is relative to the root directory. This folder is specially tracked by Azure ML in the sense that any files written to that folder during script execution on the remote target will be picked up by Run History; these files (known as artifacts) will be available as part of the run history record."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "The next cell will print out the training code for you to inspect it."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "with open(os.path.join(script_folder, './tf_mnist.py'), 'r') as f:\n    print(f.read())",
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": "# Copyright (c) Microsoft Corporation. All rights reserved.\n# Licensed under the MIT License.\n\nimport numpy as np\nimport argparse\nimport os\nimport tensorflow as tf\n\nfrom azureml.core import Run\nfrom utils import load_data\n\nprint(\"TensorFlow version:\", tf.VERSION)\n\nparser = argparse.ArgumentParser()\nparser.add_argument('--data-folder', type=str, dest='data_folder', help='data folder mounting point')\nparser.add_argument('--batch-size', type=int, dest='batch_size', default=50, help='mini batch size for training')\nparser.add_argument('--first-layer-neurons', type=int, dest='n_hidden_1', default=100,\n                    help='# of neurons in the first layer')\nparser.add_argument('--second-layer-neurons', type=int, dest='n_hidden_2', default=100,\n                    help='# of neurons in the second layer')\nparser.add_argument('--learning-rate', type=float, dest='learning_rate', default=0.01, help='learning rate')\nargs = parser.parse_args()\n\ndata_folder = os.path.join(args.data_folder, 'mnist')\n\nprint('training dataset is stored here:', data_folder)\n\nX_train = load_data(os.path.join(data_folder, 'train-images.gz'), False) / 255.0\nX_test = load_data(os.path.join(data_folder, 'test-images.gz'), False) / 255.0\n\ny_train = load_data(os.path.join(data_folder, 'train-labels.gz'), True).reshape(-1)\ny_test = load_data(os.path.join(data_folder, 'test-labels.gz'), True).reshape(-1)\n\nprint(X_train.shape, y_train.shape, X_test.shape, y_test.shape, sep='\\n')\ntraining_set_size = X_train.shape[0]\n\nn_inputs = 28 * 28\nn_h1 = args.n_hidden_1\nn_h2 = args.n_hidden_2\nn_outputs = 10\nlearning_rate = args.learning_rate\nn_epochs = 20\nbatch_size = args.batch_size\n\nwith tf.name_scope('network'):\n    # construct the DNN\n    X = tf.placeholder(tf.float32, shape=(None, n_inputs), name='X')\n    y = tf.placeholder(tf.int64, shape=(None), name='y')\n    h1 = tf.layers.dense(X, n_h1, activation=tf.nn.relu, name='h1')\n    h2 = tf.layers.dense(h1, n_h2, activation=tf.nn.relu, name='h2')\n    output = tf.layers.dense(h2, n_outputs, name='output')\n\nwith tf.name_scope('train'):\n    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=output)\n    loss = tf.reduce_mean(cross_entropy, name='loss')\n    optimizer = tf.train.GradientDescentOptimizer(learning_rate)\n    train_op = optimizer.minimize(loss)\n\nwith tf.name_scope('eval'):\n    correct = tf.nn.in_top_k(output, y, 1)\n    acc_op = tf.reduce_mean(tf.cast(correct, tf.float32))\n\ninit = tf.global_variables_initializer()\nsaver = tf.train.Saver()\n\n# start an Azure ML run\nrun = Run.get_context()\n\nwith tf.Session() as sess:\n    init.run()\n    for epoch in range(n_epochs):\n\n        # randomly shuffle training set\n        indices = np.random.permutation(training_set_size)\n        X_train = X_train[indices]\n        y_train = y_train[indices]\n\n        # batch index\n        b_start = 0\n        b_end = b_start + batch_size\n        for _ in range(training_set_size // batch_size):\n            # get a batch\n            X_batch, y_batch = X_train[b_start: b_end], y_train[b_start: b_end]\n\n            # update batch index for the next batch\n            b_start = b_start + batch_size\n            b_end = min(b_start + batch_size, training_set_size)\n\n            # train\n            sess.run(train_op, feed_dict={X: X_batch, y: y_batch})\n        # evaluate training set\n        acc_train = acc_op.eval(feed_dict={X: X_batch, y: y_batch})\n        # evaluate validation set\n        acc_val = acc_op.eval(feed_dict={X: X_test, y: y_test})\n\n        # log accuracies\n        run.log('training_acc', np.float(acc_train))\n        run.log('validation_acc', np.float(acc_val))\n        print(epoch, '-- Training accuracy:', acc_train, '\\b Validation accuracy:', acc_val)\n        y_hat = np.argmax(output.eval(feed_dict={X: X_test}), axis=1)\n\n    run.log('final_acc', np.float(acc_val))\n\n    os.makedirs('./outputs/model', exist_ok=True)\n    # files saved in the \"./outputs\" folder are automatically uploaded into run history\n    saver.save(sess, './outputs/model/mnist-tf.model')\n\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Create TensorFlow estimator\nNext, we construct an `azureml.train.dnn.TensorFlow` estimator object, use the Batch AI cluster as compute target, and pass the mount-point of the datastore to the training code as a parameter.\nThe TensorFlow estimator is providing a simple way of launching a TensorFlow training job on a compute target. It will automatically provide a docker image that has TensorFlow installed -- if additional pip or conda packages are required, their names can be passed in via the `pip_packages` and `conda_packages` arguments and they will be included in the resulting docker."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.train.dnn import TensorFlow\n\nscript_params = {\n    '--data-folder': ws.get_default_datastore().as_mount(),\n    '--batch-size': 50,\n    '--first-layer-neurons': 300,\n    '--second-layer-neurons': 100,\n    '--learning-rate': 0.01\n}\n\nest = TensorFlow(source_directory=script_folder,\n                 script_params=script_params,\n                 compute_target=compute_target,\n                 entry_script='tf_mnist.py', \n                 use_gpu=True)",
      "execution_count": 18,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Submit job to run\nCalling the `fit` function on the estimator submits the job to Azure ML for execution. Submitting the job should only take a few seconds."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "run = exp.submit(est)",
      "execution_count": 19,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Monitor the Run\nAs the Run is executed, it will go through the following stages:\n1. Preparing: A docker image is created matching the Python environment specified by the TensorFlow estimator and it will be uploaded to the workspace's Azure Container Registry. This step will only happen once for each Python environment -- the container will then be cached for subsequent runs. Creating and uploading the image takes about **5 minutes**. While the job is preparing, logs are streamed to the run history and can be viewed to monitor the progress of the image creation.\n\n2. Scaling: If the compute needs to be scaled up (i.e. the Batch AI cluster requires more nodes to execute the run than currently available), the cluster will attempt to scale up in order to make the required amount of nodes available. Scaling typically takes about **5 minutes**.\n\n3. Running: All scripts in the script folder are uploaded to the compute target, data stores are mounted/copied and the `entry_script` is executed. While the job is running, stdout and the `./logs` folder are streamed to the run history and can be viewed to monitor the progress of the run.\n\n4. Post-Processing: The `./outputs` folder of the run is copied over to the run history\n\nThere are multiple ways to check the progress of a running job. We can use a Jupyter notebook widget. \n\n**Note: The widget will automatically update ever 10-15 seconds, always showing you the most up-to-date information about the run**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.widgets import RunDetails\n\nRunDetails(run).show()",
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "77fef1346f41456e9c0cff07ca02a59f",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": "_UserRunWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': True, 'log_level': 'INFO', 's…"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "We can also periodically check the status of the run object, and navigate to Azure portal to monitor the run."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "run",
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 21,
          "data": {
            "text/html": "<table style=\"width:100%\"><tr><th>Experiment</th><th>Id</th><th>Type</th><th>Status</th><th>Details Page</th><th>Docs Page</th></tr><tr><td>tf-mnist</td><td>tf-mnist_1545830752940</td><td>azureml.scriptrun</td><td>Queued</td><td><a href=\"https://mlworkspace.azure.ai/portal/subscriptions/70b8f39e-8863-49f7-b6ba-34a80799550c/resourceGroups/resgrpAMLS/providers/Microsoft.MachineLearningServices/workspaces/AMLSworkspace/experiments/tf-mnist/runs/tf-mnist_1545830752940\" target=\"_blank\" rel=\"noopener\">Link to Azure Portal</a></td><td><a href=\"https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.script_run.ScriptRun?view=azure-ml-py\" target=\"_blank\" rel=\"noopener\">Link to Documentation</a></td></tr></table>",
            "text/plain": "Run(Experiment: tf-mnist,\nId: tf-mnist_1545830752940,\nType: azureml.scriptrun,\nStatus: Queued)"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "run.wait_for_completion(show_output=True)",
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": "RunId: tf-mnist_1545830752940\n\nStreaming azureml-logs/80_driver_log.txt\n========================================\n\nTensorFlow version: 1.10.0\ntraining dataset is stored here: /mnt/batch/tasks/shared/LS_root/jobs/amlsworkspace/azureml/tf-mnist_1545830752940/mounts/workspaceblobstore/mnist\n(60000, 784)\n(60000,)\n(10000, 784)\n(10000,)\n2018-12-26 13:33:43.256414: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA\n2018-12-26 13:33:43.365578: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1405] Found device 0 with properties: \nname: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\npciBusID: e96d:00:00.0\ntotalMemory: 11.17GiB freeMemory: 11.10GiB\n2018-12-26 13:33:43.365619: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1484] Adding visible gpu devices: 0\n2018-12-26 13:33:43.638617: I tensorflow/core/common_runtime/gpu/gpu_device.cc:965] Device interconnect StreamExecutor with strength 1 edge matrix:\n2018-12-26 13:33:43.638667: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971]      0 \n2018-12-26 13:33:43.638679: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] 0:   N \n2018-12-26 13:33:43.638931: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1097] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10758 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: e96d:00:00.0, compute capability: 3.7)\n0 -- Training accuracy: 0.92 Validation accuracy: 0.9086\n1 -- Training accuracy: 0.94 Validation accuracy: 0.9258\n2 -- Training accuracy: 0.9 Validation accuracy: 0.9325\n3 -- Training accuracy: 0.84 Validation accuracy: 0.941\n4 -- Training accuracy: 0.92 Validation accuracy: 0.9452\n5 -- Training accuracy: 0.96 Validation accuracy: 0.95\n6 -- Training accuracy: 1.0 Validation accuracy: 0.9531\n7 -- Training accuracy: 1.0 Validation accuracy: 0.957\n8 -- Training accuracy: 0.98 Validation accuracy: 0.9594\n9 -- Training accuracy: 0.94 Validation accuracy: 0.9617\n10 -- Training accuracy: 0.96 Validation accuracy: 0.9628\n11 -- Training accuracy: 0.96 Validation accuracy: 0.9653\n12 -- Training accuracy: 0.96 Validation accuracy: 0.9664\n13 -- Training accuracy: 0.98 Validation accuracy: 0.9678\n14 -- Training accuracy: 0.98 Validation accuracy: 0.9697\n15 -- Training accuracy: 1.0 Validation accuracy: 0.9697\n16 -- Training accuracy: 0.96 Validation accuracy: 0.9715\n17 -- Training accuracy: 1.0 Validation accuracy: 0.9721\n18 -- Training accuracy: 1.0 Validation accuracy: 0.9721\n19 -- Training accuracy: 0.98 Validation accuracy: 0.9719\n\n\nThe experiment completed successfully. Finalizing run...\nLogging experiment finalizing status in history service\n\nExecution Summary\n=================\nRunId: tf-mnist_1545830752940\n\n",
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "execution_count": 22,
          "data": {
            "text/plain": "{'runId': 'tf-mnist_1545830752940',\n 'target': 'gpucluster',\n 'status': 'Finalizing',\n 'startTimeUtc': '2018-12-26T13:31:43.316712Z',\n 'properties': {'azureml.runsource': 'experiment',\n  'ContentSnapshotId': '67d43dbe-7c3e-429b-bdac-29a14f174304'},\n 'runDefinition': {'Script': 'tf_mnist.py',\n  'Arguments': ['--data-folder',\n   '$AZUREML_DATAREFERENCE_workspaceblobstore',\n   '--batch-size',\n   '50',\n   '--first-layer-neurons',\n   '300',\n   '--second-layer-neurons',\n   '100',\n   '--learning-rate',\n   '0.01'],\n  'SourceDirectoryDataStore': None,\n  'Framework': 0,\n  'Communicator': 0,\n  'Target': 'gpucluster',\n  'DataReferences': {'workspaceblobstore': {'DataStoreName': 'workspaceblobstore',\n    'Mode': 'Mount',\n    'PathOnDataStore': None,\n    'PathOnCompute': None,\n    'Overwrite': False}},\n  'JobName': None,\n  'AutoPrepareEnvironment': True,\n  'MaxRunDurationSeconds': None,\n  'NodeCount': 1,\n  'Environment': {'Python': {'InterpreterPath': 'python',\n    'UserManagedDependencies': False,\n    'CondaDependencies': {'name': 'project_environment',\n     'dependencies': ['python=3.6.2',\n      {'pip': ['azureml-defaults', 'tensorflow-gpu==1.10.0']}]},\n    'CondaDependenciesFile': None},\n   'EnvironmentVariables': {'EXAMPLE_ENV_VAR': 'EXAMPLE_VALUE',\n    'NCCL_SOCKET_IFNAME': '^docker0'},\n   'Docker': {'BaseImage': 'mcr.microsoft.com/azureml/base-gpu:0.2.0',\n    'Enabled': True,\n    'SharedVolumes': True,\n    'Preparation': None,\n    'GpuSupport': True,\n    'Arguments': [],\n    'BaseImageRegistry': {'Address': None,\n     'Username': None,\n     'Password': None}},\n   'Spark': {'Repositories': ['https://mmlspark.azureedge.net/maven'],\n    'Packages': [{'Group': 'com.microsoft.ml.spark',\n      'Artifact': 'mmlspark_2.11',\n      'Version': '0.12'}],\n    'PrecachePackages': True}},\n  'History': {'OutputCollection': True},\n  'Spark': {'Configuration': {'spark.app.name': 'Azure ML Experiment',\n    'spark.yarn.maxAppAttempts': '1'}},\n  'BatchAi': {'NodeCount': 0},\n  'AmlCompute': {'Name': None,\n   'VmSize': None,\n   'VmPriority': None,\n   'RetainCluster': False,\n   'ClusterMaxNodeCount': 1},\n  'Tensorflow': {'WorkerCount': 1, 'ParameterServerCount': 1},\n  'Mpi': {'ProcessCountPerNode': 1},\n  'Hdi': {'YarnDeployMode': 2},\n  'ContainerInstance': {'Region': None, 'CpuCores': 0, 'MemoryGb': 0},\n  'ExposedPorts': None,\n  'PrepareEnvironment': None},\n 'logFiles': {'azureml-logs/60_control_log.txt': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545830752940/azureml-logs/60_control_log.txt?sv=2018-03-28&sr=b&sig=ELOHsgBKk9ertkttkHM7heJRCwAhNOu9bqaupHTGCwQ%3D&st=2018-12-26T13%3A24%3A24Z&se=2018-12-26T21%3A34%3A24Z&sp=r',\n  'azureml-logs/80_driver_log.txt': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545830752940/azureml-logs/80_driver_log.txt?sv=2018-03-28&sr=b&sig=tSgoPhHH1%2BMuOeGvNiboRZM3LBi6h47SBNoyFIusJqg%3D&st=2018-12-26T13%3A24%3A24Z&se=2018-12-26T21%3A34%3A24Z&sp=r',\n  'azureml-logs/azureml.log': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545830752940/azureml-logs/azureml.log?sv=2018-03-28&sr=b&sig=cS1jDymGRZiR0iC7E943ZOJ6HNIXBqquw9C8nKBLQPY%3D&st=2018-12-26T13%3A24%3A24Z&se=2018-12-26T21%3A34%3A24Z&sp=r'}}"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### The Run object\nThe Run object provides the interface to the run history -- both to the job and to the control plane (this notebook), and both while the job is running and after it has completed. It provides a number of interesting features for instance:\n* `run.get_details()`: Provides a rich set of properties of the run\n* `run.get_metrics()`: Provides a dictionary with all the metrics that were reported for the Run\n* `run.get_file_names()`: List all the files that were uploaded to the run history for this Run. This will include the `outputs` and `logs` folder, azureml-logs and other logs, as well as files that were explicitly uploaded to the run using `run.upload_file()`\n\nBelow are some examples -- please run through them and inspect their output. "
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "run.get_details()",
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 33,
          "data": {
            "text/plain": "{'runId': 'tf-mnist_1545830752940',\n 'target': 'gpucluster',\n 'status': 'Completed',\n 'startTimeUtc': '2018-12-26T13:31:43.316712Z',\n 'endTimeUtc': '2018-12-26T13:34:29.585771Z',\n 'properties': {'azureml.runsource': 'experiment',\n  'ContentSnapshotId': '67d43dbe-7c3e-429b-bdac-29a14f174304'},\n 'runDefinition': {'Script': 'tf_mnist.py',\n  'Arguments': ['--data-folder',\n   '$AZUREML_DATAREFERENCE_workspaceblobstore',\n   '--batch-size',\n   '50',\n   '--first-layer-neurons',\n   '300',\n   '--second-layer-neurons',\n   '100',\n   '--learning-rate',\n   '0.01'],\n  'SourceDirectoryDataStore': None,\n  'Framework': 0,\n  'Communicator': 0,\n  'Target': 'gpucluster',\n  'DataReferences': {'workspaceblobstore': {'DataStoreName': 'workspaceblobstore',\n    'Mode': 'Mount',\n    'PathOnDataStore': None,\n    'PathOnCompute': None,\n    'Overwrite': False}},\n  'JobName': None,\n  'AutoPrepareEnvironment': True,\n  'MaxRunDurationSeconds': None,\n  'NodeCount': 1,\n  'Environment': {'Python': {'InterpreterPath': 'python',\n    'UserManagedDependencies': False,\n    'CondaDependencies': {'name': 'project_environment',\n     'dependencies': ['python=3.6.2',\n      {'pip': ['azureml-defaults', 'tensorflow-gpu==1.10.0']}]},\n    'CondaDependenciesFile': None},\n   'EnvironmentVariables': {'EXAMPLE_ENV_VAR': 'EXAMPLE_VALUE',\n    'NCCL_SOCKET_IFNAME': '^docker0'},\n   'Docker': {'BaseImage': 'mcr.microsoft.com/azureml/base-gpu:0.2.0',\n    'Enabled': True,\n    'SharedVolumes': True,\n    'Preparation': None,\n    'GpuSupport': True,\n    'Arguments': [],\n    'BaseImageRegistry': {'Address': None,\n     'Username': None,\n     'Password': None}},\n   'Spark': {'Repositories': ['https://mmlspark.azureedge.net/maven'],\n    'Packages': [{'Group': 'com.microsoft.ml.spark',\n      'Artifact': 'mmlspark_2.11',\n      'Version': '0.12'}],\n    'PrecachePackages': True}},\n  'History': {'OutputCollection': True},\n  'Spark': {'Configuration': {'spark.app.name': 'Azure ML Experiment',\n    'spark.yarn.maxAppAttempts': '1'}},\n  'BatchAi': {'NodeCount': 0},\n  'AmlCompute': {'Name': None,\n   'VmSize': None,\n   'VmPriority': None,\n   'RetainCluster': False,\n   'ClusterMaxNodeCount': 1},\n  'Tensorflow': {'WorkerCount': 1, 'ParameterServerCount': 1},\n  'Mpi': {'ProcessCountPerNode': 1},\n  'Hdi': {'YarnDeployMode': 2},\n  'ContainerInstance': {'Region': None, 'CpuCores': 0, 'MemoryGb': 0},\n  'ExposedPorts': None,\n  'PrepareEnvironment': None},\n 'logFiles': {'azureml-logs/60_control_log.txt': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545830752940/azureml-logs/60_control_log.txt?sv=2018-03-28&sr=b&sig=4GNhhvAneeT9c7xiQqRmmS9fy7gxYPB4J7aG75GpRXE%3D&st=2018-12-26T13%3A25%3A17Z&se=2018-12-26T21%3A35%3A17Z&sp=r',\n  'azureml-logs/80_driver_log.txt': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545830752940/azureml-logs/80_driver_log.txt?sv=2018-03-28&sr=b&sig=s5pLNcO50cVzoeQcmNmCq1U3GNL6qU%2BvtAXLAD47sjc%3D&st=2018-12-26T13%3A25%3A17Z&se=2018-12-26T21%3A35%3A17Z&sp=r',\n  'azureml-logs/azureml.log': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545830752940/azureml-logs/azureml.log?sv=2018-03-28&sr=b&sig=vtvuwj8C5Su9ZnKAOGVJN9Ipq9B9oH6C%2FHpDSeiRlCM%3D&st=2018-12-26T13%3A25%3A17Z&se=2018-12-26T21%3A35%3A17Z&sp=r',\n  'azureml-logs/55_batchai_execution.txt': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545830752940/azureml-logs/55_batchai_execution.txt?sv=2018-03-28&sr=b&sig=zMnlSNRlQRENutQ8dRngPuj5i0cDsI3aTfLIIe%2FYjHs%3D&st=2018-12-26T13%3A25%3A17Z&se=2018-12-26T21%3A35%3A17Z&sp=r'}}"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "run.get_metrics()",
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 24,
          "data": {
            "text/plain": "{'training_acc': [0.9200000166893005,\n  0.9399999976158142,\n  0.8999999761581421,\n  0.8399999737739563,\n  0.9200000166893005,\n  0.9599999785423279,\n  1,\n  1,\n  0.9800000190734863,\n  0.9399999976158142,\n  0.9599999785423279,\n  0.9599999785423279,\n  0.9599999785423279,\n  0.9800000190734863,\n  0.9800000190734863,\n  1,\n  0.9599999785423279,\n  1,\n  1,\n  0.9800000190734863],\n 'validation_acc': [0.9085999727249146,\n  0.9258000254631042,\n  0.9325000047683716,\n  0.9409999847412109,\n  0.9452000260353088,\n  0.949999988079071,\n  0.9531000256538391,\n  0.9570000171661377,\n  0.9593999981880188,\n  0.9617000222206116,\n  0.9628000259399414,\n  0.9653000235557556,\n  0.9664000272750854,\n  0.9678000211715698,\n  0.9696999788284302,\n  0.9696999788284302,\n  0.9714999794960022,\n  0.972100019454956,\n  0.972100019454956,\n  0.9718999862670898],\n 'final_acc': 0.9718999862670898}"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "run.get_file_names()",
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "execution_count": 36,
          "data": {
            "text/plain": "['azureml-logs/60_control_log.txt',\n 'azureml-logs/80_driver_log.txt',\n 'outputs/model/checkpoint',\n 'outputs/model/mnist-tf.model.data-00000-of-00001',\n 'outputs/model/mnist-tf.model.index',\n 'outputs/model/mnist-tf.model.meta',\n 'driver_log',\n 'azureml-logs/azureml.log',\n 'acc_over_epochs.png.png',\n 'azureml-logs/55_batchai_execution.txt']"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Plot accuracy over epochs\nSince we can retrieve the metrics from the run, we can easily make plots using `matplotlib` in the notebook. Then we can add the plotted image to the run using `run.log_image()`, so all information about the run is kept together."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import os\n\nos.makedirs('./imgs', exist_ok=True)\nmetrics = run.get_metrics()\n\nplt.figure(figsize = (13,5))\nplt.plot(metrics['validation_acc'], 'r-', lw=4, alpha=.6)\nplt.plot(metrics['training_acc'], 'b--', alpha=0.5)\nplt.legend(['Full evaluation set', 'Training set mini-batch'])\nplt.xlabel('epochs', fontsize=14)\nplt.ylabel('accuracy', fontsize=14)\nplt.title('Accuracy over Epochs', fontsize=16)\nrun.log_image(name='acc_over_epochs.png', plot=plt)\nplt.show()",
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "error",
          "ename": "AzureMLException",
          "evalue": "UserError: Resource Conflict: ArtifactId ExperimentRun/tf-mnist_1545830752940/acc_over_epochs.png.png already exists.",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAzureMLException\u001b[0m                          Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-37-7c003a373bfd>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mylabel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'accuracy'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfontsize\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m14\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtitle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Accuracy over Epochs'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfontsize\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m16\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mrun\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlog_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'acc_over_epochs.png'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplot\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mplt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m~/anaconda3_501/lib/python3.6/site-packages/azureml/core/run.py\u001b[0m in \u001b[0;36mlog_image\u001b[0;34m(self, name, path, plot)\u001b[0m\n\u001b[1;32m    888\u001b[0m         \u001b[0;34m:\u001b[0m\u001b[0mtype\u001b[0m \u001b[0mplot\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mmatplotlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpyplot\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    889\u001b[0m         \"\"\"\n\u001b[0;32m--> 890\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_client\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlog_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpath\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplot\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mplot\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    891\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    892\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mupload_file\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpath_or_stream\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m~/anaconda3_501/lib/python3.6/site-packages/azureml/_run_impl/run_history_facade.py\u001b[0m in \u001b[0;36mlog_image\u001b[0;34m(self, name, path, plot)\u001b[0m\n\u001b[1;32m    243\u001b[0m             artifact_path = (self._save_img_path(path)\n\u001b[1;32m    244\u001b[0m                              \u001b[0;32mif\u001b[0m \u001b[0mpath\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 245\u001b[0;31m                              else self._save_mpl_plot(name, plot))\n\u001b[0m\u001b[1;32m    246\u001b[0m             \u001b[0maml_artifact_uri\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"aml://artifactId/{0}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martifact_path\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    247\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmetrics\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_log_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maml_artifact_uri\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m~/anaconda3_501/lib/python3.6/site-packages/azureml/_run_impl/run_history_facade.py\u001b[0m in \u001b[0;36m_save_mpl_plot\u001b[0;34m(self, name, plot)\u001b[0m\n\u001b[1;32m    214\u001b[0m             \u001b[0mplot\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msavefig\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstream\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mformat\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mext\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    215\u001b[0m             \u001b[0mstream\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mseek\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 216\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_upload_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martifact_path\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstream\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mext\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    217\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mAttributeError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mattribute_error\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    218\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mAzureMLException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Invalid plot, must be matplotlib.pyplot\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minner_exception\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mattribute_error\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m~/anaconda3_501/lib/python3.6/site-packages/azureml/_run_impl/run_history_facade.py\u001b[0m in \u001b[0;36m_upload_image\u001b[0;34m(self, path, artifact, ext)\u001b[0m\n\u001b[1;32m    231\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_upload_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0martifact\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mext\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    232\u001b[0m         return self.artifacts.upload_artifact(artifact, self._origin, self._run_id, path,\n\u001b[0;32m--> 233\u001b[0;31m                                               content_type=\"image/{}\".format(ext))\n\u001b[0m\u001b[1;32m    234\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    235\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mlog_image\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpath\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mplot\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m~/anaconda3_501/lib/python3.6/site-packages/azureml/_restclient/artifacts_client.py\u001b[0m in \u001b[0;36mupload_artifact\u001b[0;34m(self, artifact, *args, **kwargs)\u001b[0m\n\u001b[1;32m    116\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martifact\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIOBase\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    117\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_logger\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdebug\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Uploading io artifact\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 118\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupload_artifact_from_stream\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0martifact\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    119\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    120\u001b[0m             raise UserErrorException(\"UnsupportedType: type {} is invalid, \"\n",
            "\u001b[0;32m~/anaconda3_501/lib/python3.6/site-packages/azureml/_restclient/artifacts_client.py\u001b[0m in \u001b[0;36mupload_artifact_from_stream\u001b[0;34m(self, stream, origin, container, name, content_type, session)\u001b[0m\n\u001b[1;32m     94\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     95\u001b[0m         \u001b[0;31m# Construct body\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 96\u001b[0;31m         \u001b[0mres\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcreate_empty_artifacts\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0morigin\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontainer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     97\u001b[0m         \u001b[0martifact\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mres\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0martifacts\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     98\u001b[0m         \u001b[0mcontent_information\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mres\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0martifact_content_information\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m~/anaconda3_501/lib/python3.6/site-packages/azureml/_restclient/artifacts_client.py\u001b[0m in \u001b[0;36mcreate_empty_artifacts\u001b[0;34m(self, origin, container, paths)\u001b[0m\n\u001b[1;32m     78\u001b[0m                 error_messages.append(\"{}: {}\".format(error.code,\n\u001b[1;32m     79\u001b[0m                                                       error.message))\n\u001b[0;32m---> 80\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mAzureMLException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"\\n\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merror_messages\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     81\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     82\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mres\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAzureMLException\u001b[0m: UserError: Resource Conflict: ArtifactId ExperimentRun/tf-mnist_1545830752940/acc_over_epochs.png.png already exists."
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAFTCAYAAACktJpRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3XecVeW1+P/PmgIzDG2GXqT3XoYmFqQr2MBYUTAqemPNveZGv/Em+Zncq9/Em2+iklgQUTGIihKiRqQIKkgZYOggVRh6ZygDU57fH+sMc+bMmX7mlJn1fr3265S9z97rnDNlr/08z3rEOYcxxhhjjDHGlEdUqAMwxhhjjDHGRD5LLIwxxhhjjDHlZomFMcYYY4wxptwssTDGGGOMMcaUmyUWxhhjjDHGmHKzxMIYY4wxxhhTbpZYGGNMKYjIVBFxIvKnUMdiAkNEpnu+U3/LnBDHtkdEZoQyBmOMKamYUAdgjDGRQkTigZ94Ht4jIv/pnMsKZUwmYI4CN/l5/kSwAzHGmEhliYUxxpTcrUBt4AvgBmA08FlII/JDRKo75y6GOo5wUoLP5JJzbnnQAjLGmErIukIZY0zJTQROApOAC8B9/jYSkZ4i8qmIHBeRCyKyTUSe9dnmVhFZKiJnReSMiKwUkZs861p5uuFM8nnNEM/zQ7yeWywi34nIjSKyVkQuAj/zrHtMRL4XkRMickpElovIGD/xJojIiyKyU0QuisghEZktIo1EpK/nmDf7ed10EUkTkejCPjBRP/d8BpdE5KCIvCoitb222SQis/28doDn2Lf4fLZzReSk57NdKiJXFxLXIBFZJiIXgD8UFmNJee33ShFZJSIZnq5Kj/vZtr+ILPB8v+dEZKGI9Pez3bUiMl9ETnu2WyciD/jZ7k4R2eLZJkVErvJZ38+zn+Micl5EdonIX8v7no0xpjQssTDGmBIQkabAcGCWc+4oMAe4SUQSfbbrD3wPtAV+DowB/gQ099rmceAT4AiarPwE+BRoVcbwOgAvA68Ao4CFnudbAVM9+78DSAE+E5HrvWKpBswHngCmA2OBx9AuQInOudXAKuBhn/dZF7gdmOqcyy4itv9G3/984Eb0BH8S8LmI5P4Peg8Y6/tZAhM8cXzhOWYfYBmQBDwEjAeOAwtEpK/Pa+sAHwAzgeuBvxcRY+57ivGziM9mtYFZwDvALcBi4GXvJFBEegBLgETPe73P87olItLTa7ub0e+qGvr53gxMA1r6HPNq4D+A/0K/x2j0e6zr2U9NYB6Q7TneDcDzWK8EY0ywOedsscUWW2wpZgF+CThgkOfxKM/jR3y2+wbYB9QoZD+1gXTgkyKO1cqz70k+zw/xPD/E67nFQA7Qq5j4o9ATza+Af3g9/1PPPm8q4rWT0JPWll7PPQFkAc2LeF0SkAFM93l+gvcxgSs8+3/Ya5tYdNzDX72eWwhsAap5PRfteW6O13PTPfu/uYTfbe72/pan/Wx3p8/r5wM/AuJ5/DFwCqjr872fyP3eAQH2oMleVBGx7UFbyRK9nkv2xHG3z+Meof49scUWW6r2Yi0WxhhTMvcB251z33seLwAO4NUdSkRqAIOB951z5wvZz5VATeCNAMa2xzmX6vukpxvTZyJyGE0CMoERQEevzUYCh5xzc4vY/wfoifJDXs89DHzunEsr4nUDgeqAb1WjDzzxXAvgnNuHXuG/12ub0UB94F3Pe4n3bP8RkJPbooCeoC8ArvE5RhalG/9yBOjnZ3nPZ7tswLfb1gdAC6CZ5/E1wGfOuVO5GzjnzgBzc98z+h20RFt8coqJ7Xvn3Emvxxs8ty08t9vR7+d1EZkgIlcUsz9jjKkQllgYY0wxRKQf0AX4RETqerqg1EK7Mw0SkQ6eTRPRv6tFnWzX89wWtU1pHfR9wnNyuRBtNXgcTWj6AV8CcT7x7C9q5865DOBt4AHPCf3V6OfxWjFxJfmLz2klreNe60ETiMEi0trz+F5gh8sbUJ2Etk78F5ogeS+PAYleXasAjriiu2j5ynTOpfhZDvtsd9I5l+nzXO42uYlFku979jiE/oxA6X4O8lWmcnmD0OM8j08D16GJ7l+BvSKyUUTGl2DfxhgTMJZYGGNM8SZ6bn+JdkvJXR7zPJ/banES7ZbUjMId89wWtU2G57aaz/P1fDf0cH6eG42OM7jdOfehc265cy4FqOEnnqJiyfU3oBE6DuBhtIvOvGJek3tC3Nj7SU9LQz00ucg1GzgPTPAM7L6R/K0Fp9DP9hX8tyz087ny7+8zCYREEYn1ea6R5zY3QTuBz3v2aEzeZ1KSn4MSc86lOufGo0nNIGAn8KGIdAvE/o0xpiQssTDGmCJ4BjffCaxArwr7LqnAvSIinu5P36Enx/GF7HIZcBaYXMRhDwMXAd+TwgIVnYqQm0BcvrruaVkZ7LPdV0BjEbmxqJ0553Z6tv0FcBvwZgm68CxH38edPs/fgY73WOK1/3TgH2hLxU/Qq/Hvea0/B3wL9ATW+GtdKCaWQIlGB417uxPYS15isQQYIyK1cjfw3L+RvPf8A5qcPehngHiZOeeyPK08/4X+j+8cqH0bY0xxrGKEMcYUbSx6df0/nHOLfVeKyOvo1fwhwNfA0+jJ4/ci8r9oV5c26ODqx51z6aKlZ1/xlFh9Hx3M3QvIcM694pxzIjIL7Xr0A7ANTSqGlCLuBeg4g3c9cTQB/j/0BNj7otIMdOzETBF5AU2gaqGD0//snNvqte1f0ZP/TLR6UZGccydEZyh/VkTOodWdOgO/RxOwz31e8i5wlyfO75xzu33W/zs6OH6eiLyFdjeqD/QBop1zzxQXUxGqichAP8+fd86t93qcDvxBROqjYxvuQquFTXLO5baS/A79uVkoIv8XbT35JZrsPQ/g+Y6fQrvTLRKR19DB6p2Bhs6535Q0cBEZiyaqc4DdQAI6uD4drVBmjDHBEerR47bYYost4bygJ9JnKLzKUx20C890r+d6A/9Eu+9cALYCv/R53W3oSfwFz/5XAGO91tdFr9gfQ7vPvIYmF/6qQn1XSGy3e46dAWxCr6xPRwd7e29XE/gjWtnoEnrC/jF6guu9XTRwDvioFJ+foGV3t3ntewpQ28+20Z71DphcyP46o4Olj6CtIWnooOgbvLaZDqSVIsbpFF4VaqPvftHxKqs8n+uPwBN+9jkATe7Oej6zhUB/P9sNRRPSs55lHXC/1/o9wAw/r3PAbz33O6IlcHd7YjqKJnEDQv37Y4sttlStJbc0njHGGFMkERmBdoca7pxbWNz2lY2ITEffe/PitjXGmKrIukIZY4wpkoi0Rbtz/T90fEOVSyqMMcYUzwZvG2OMKc5/Af9Cux7dV8y2xhhjqijrCmWMMcYYY4wpN2uxMMYYY4wxxpSbJRbGGGOMMcaYcqu0g7fr16/vWrVqFeowjDHGGGOMiWirV68+5pxrUNx2lTaxaNWqFSkpwZqI1RhjjDHGmMpJRH4syXbWFcoYY4wxxhhTbpZYGGOMMcYYY8rNEgtjjDHGGGNMuVXaMRb+ZGZmkpaWRkZGRqhDMZVQXFwczZs3JzY2NtShGGOMMcYEXZVKLNLS0qhVqxatWrVCREIdjqlEnHMcP36ctLQ0WrduHepwjDHGGGOCrkp1hcrIyKBevXqWVJiAExHq1atnrWHGGGOMqbKqVGIBWFJhKoz9bBljjDGmKgtaYiEi00TkiIhsLGS9iMjLIrJDRNaLSB+vdRNFZLtnmRismCtCdHQ0vXr1urzs2bOnyO1btWrFsWPHAKhZs2aFxuZ9rNKaM2cOmzdvvvz417/+NQsWLAhUaCX2P//zP0E/pjHGGGOMCW6LxXRgdBHrrwfae5bJwN8ARCQJ+A0wAOgP/EZEEis00goUHx9Pamrq5aWyzA7um1g8//zzDB8+POhxWGJhjDHGGBMaQRu87Zz7RkRaFbHJzcC7zjkHLBeRuiLSBBgCzHfOnQAQkflogjKzYiMOnunTp5OSksKrr74KwNixY3n66acZMmRIiV4/Y8YMXn75ZS5dusSAAQP461//yhtvvMHu3bv5wx/+cPkYq1ev5pVXXuGWW25h3759ZGRk8OSTTzJ58uR8+9uzZw9jx45l40ZtXHrppZc4e/Ysv/3tb3nzzTd54403uHTpEu3ateO9994jNTWVuXPnsmTJEn7/+98ze/Zsfve73zF27Fhuu+02Fi5cyNNPP01WVhb9+vXjb3/7G9WrV6dVq1ZMnDiRf/7zn2RmZvLRRx/RqVOnfLFs2rSJ+++/n0uXLpGTk8Ps2bNp37693/f8q1/9igsXLtCrVy+6du3K+++/X85vxoTSrl1w8mTe4+rVoVs3vb99O5w5k3/7+Hjo0kXvb90K587lX1+zJnTsqPc3bQLf4TC1a0P79np/wwa4dClvXY0a0KkTWG83Y4ypeGlpcPhwwef79NG/w3v3wtGj+ddFRUHv3np/9244cSL/+pgY6NlT7+/cCadO5V/v/T/mhx/0ccuW5X8vVU04VYVqBuzzepzmea6w5wsQkcloawctWrQo/EgPP1y+SIvz+uuFrso98QVo3bo1n376abkOtWXLFmbNmsXSpUuJjY3lZz/7Ge+//z633XYbgwYNupxYzJo1i1/96lcATJs2jaSkJC5cuEC/fv0YP3489erVK9Hxxo0bx0MPPQTAc889x1tvvcXjjz/OTTfddDmR8JaRkcGkSZNYuHAhHTp04L777uNvf/sbTz31FAD169dnzZo1/PWvf+Wll15i6tSp+V7/2muv8eSTT3LPPfdw6dIlsrOzC33PL774Iq+++iqpqanl+kxN6O3cCe+9l/+5xMS8P/rff6+Jh7dGjfISi+++039M3q64Ii+xWLy44D+ldu3yEosFC+D06fzrBw2CUaPK9HaMMeYy5/Tk+MAB/bsWHx/qiMLDxo1Qvz40bgxbtsDSpQW36d1bP7sNG2DVqvzrYmLyEou1a2H9+vzrExLyEotVq/QClDfv/zHLl+v/mOHDYfBgu6hUGuGUWPj72lwRzxd80rk3gDcAkpOT/W4TarldoQJl4cKFrF69mn79+gGauDRs2JAGDRrQpk0bli9fTvv27dm2bRuDBw8G4OWXX76c0Ozbt4/t27eXOLHYuHEjzz33HKdOneLs2bOMKuZMa9u2bbRu3ZoOHToAMHHiRKZMmXI5sRg3bhwAffv25ZNPPinw+kGDBvHf//3fpKWlMW7cONq3b1/oezaVR/Xq0LkzjB6d9wc9yqvj5m23QVZW/td4r7/rLsjOzr8+Ojrv/sSJkJOTf32M11/Dhx7Kv37jxrykwxhjymrNmrwT1qlTYeBAGDky1FGFXno6zJ2rF3huvx2uvhoGDCi4Xe7/g2HDdJvCXH+9fsb+Xgtw001www3513v/Dxk/Hr78Ui8ynTmj/4uiqly5o7IJp8QiDbjC63Fz4IDn+SE+zy8OWlRBEBMTQ47XWUxpSpY655g4cSIvvPBCgXV33HEHH374IZ06deLWW29FRFi8eDELFizg+++/p0aNGgwZMqTA8YqKZ9KkScyZM4eePXsyffp0Fi9eXGx8RalevTqgg9qzfM8UgbvvvpsBAwbw+eefM2rUKKZOnVrkezaVQ/PmcMcdha+vUaPo1yckFL2+uDoIvuuvvFJvndPWkJ49teuUMcaUhHOwZIm2lrZrl3f1fMUKSE6GpKRQRxhaX3+tF4Nyk4G4OF0KU9z6+PiiW4JK8j9k3DioVQuWLdPE57bb8l+gMv6FU/41F7jPUx1qIHDaOXcQmAeMFJFEz6DtkZ7nKo1WrVqRmppKTk4O+/btY+XKlSV+7bBhw/j44485cuQIACdOnODHH38EtDVgzpw5zJw5kzs8Z2mnT58mMTGRGjVqsHXrVpYvX15gn40aNeLIkSMcP36cixcv8tlnn11el56eTpMmTcjMzMw3hqFWrVqkp6cX2FenTp3Ys2cPO3bsAOC9997j2muvLfH727VrF23atOGJJ57gpptuYv369UW+59jYWDIzM0u8fxNeLl7UK0Tnz4c6Ev9OnYJvv4W33irYlcoYY/zJyYF//lOTil69tEU1NhaGDtUT1RAUUAwrhw5p16X+/cMrwRLR1qTRozXBsBaLkglai4WIzERbHuqLSBpa6SkWwDn3GvAFcAOwAzgP3O9Zd0JEfgfk9qZ7Pncgd5kVMQYiFAYPHkzr1q3p3r073bp1o0+fPsW/yKNLly78/ve/Z+TIkeTk5BAbG8uUKVNo2bIliYmJdOnShc2bN9O/f38ARo8ezWuvvUaPHj3o2LEjAwcOLLDP2NhYfv3rXzNgwABat26db0D17373OwYMGEDLli3p3r375WTizjvv5KGHHuLll1/m448/vrx9XFwcb7/9Nj/5yU8uD95+5JFHSvz+Zs2axYwZM4iNjaVx48b8+te/JikpqdD3PHnyZHr06EGfPn1s8HYE+u47XTp3Lv6KUigkJsL998P772tycdddNrjPGFO0OXO0v/+118KQIXldcmrV0v77X3+tg5GLGhpaWTkHX32lrQ/XXBPqaPwbODBvXMzRo5oU1q0b6qjClxTXVSVSJScnu5SUlHzPbdmyhc6dO4coIlMV2M9Y2Z06Ba++qgOwPUNvwtapUzBjht6OH6+JkDHG+JNbwahv34LrMjP1WudVV2lrRlWTk6MtObVra5ewcOYcvPaatqjfc48OMq9KRGS1c67Yb8kadowxYWHhQr0dNiy0cZRE3brw059Cs2bW59YYU9CJE3lVi1q08J9UgF79/tnPqmZSAdq9aOjQ8E8qQFssxo/X27ffLliZ0ChLLIwxIbd/v5YPvPJKqFMn1NGUTI0aMGkSeAqekZamV7SMMVXb/v3aVfLrr0s2XiwqSv92bNmiLRhVxcaNsG1bZP3dbNgQHnxQ/0+9/77+3zL5WWJhjAm5GjX0ip2nInLEyO0rfeiQnkh8+mnBMrfGmKpj+3aYPh2qVYMHHij5WLGDB2HWLJ2jpyrIyIAvvtD5IiJN7draYn3FFbB6dWQlRsEQTuVmjTFVVGIi3HJLqKMou0aNtDl/4UI4e1ZL5XoqKRtjqoi1a7X6U6NG2ge/uLLW3po21bFa332ns0uX5rWR6Ntv4cIFrboUiZPPxcXBhAk6n5KIVjSMjbXKUWAtFsaYEMrK0n/Ex4+HOpLyEdHJmm65Bfbs0f63fqovG2MqMeegTRvtIlmWxGD4cG3xXLQo4KGFlZMntaWiZ09o0iTU0ZRdTIwmGNnZ8Pe/w8cfF5y4tSqyxMIYEzIrV2pT8qlToY4kMHr1grvv1oGbmzeHOhpjTEXLydFuTKAtDffcU/bWynr1dC6HtWvh8OHAxRhuFizIG7RdGURHQ6dO+jf/vfe0JaYqs8QiiI4fP06vXr3o1asXjRs3plmzZpcfX7p0qUT7uP/++9m2bVuR20yZMiUkczgsWrTI74R7pVGe97djxw56lbK0xrRp0zh06FCR20ydOpWnnnqqVPs1xTt3Tmeibd8e2rYNdTSB066dVnnxTB1TpQZjGlOVXLoEH3wA06bB6dP6XHm79VxzjXalOneu/PGFq3bttHWmdu1QRxI4gwbpzNxpafl/HqoiG2MRRPXq1SM1NRWA3/72t9SsWZOnn3463zbOOZxzRBXSUe/tt98u9jiPPvpo+YMtg0WLFlG/fn2/k+6VVLDf37Rp0+jTpw+Nq1pB6jCwZImedI8cGepIAi938qSjR+Hdd+GGG2yuC2Mqk3PnYOZMrQB1ww2Bq2YXHw8PPxyZ4w5KqnfvUEdQMbp1g4QETTY//lgHeFfm77Ew1mIRBnbs2EG3bt145JFH6NOnDwcPHmTy5MkkJyfTtWtXnn/++cvbXnXVVaSmppKVlUXdunV55pln6NmzJ4MGDeLIkSMAPPfcc/z5z3++vP0zzzxD//796dixI8uWLQPg3LlzjB8/np49e3LXXXeRnJx8Oenx9otf/IIuXbrQo0cPfvnLXwJw+PBhxo0bR3JyMv3792f58uXs3LmTqVOn8sc//pFevXpdPk6u5557jkmTJjFy5EhatWrFnDlz+I//+A+6devGmDFjyPJ0TCzt+/OVmZnJvffeS/fu3bn99tu54GmT/M1vfkO/fv0uf87OOWbNmkVqaip33HHH5VajFStWMGjQIHr27MmAAQM476kVmJaWxqhRo2jfvj3PPvts2b5oc9nRo5CSorXdGzQIdTQVJyFBTzg+/FC7fRljIt+JE1oF7tAhLdTQr19g9y+iffVTUipXlbktW2DFCu0+Vlm1bq0JxU03Vc2kAiyxCBubN2/mgQceYO3atTRr1owXX3yRlJQU1q1bx/z589nsp8P26dOnufbaa1m3bh2DBg1i2rRpfvftnGPlypX88Y9/vJykvPLKKzRu3Jh169bxzDPPsHbt2gKvO3z4MF988QWbNm1i/fr1l0+on3jiCf7zP/+TlJQUPvzwQx588EHatm3Lgw8+yC9+8QtSU1O58sorC+xv9+7dfPHFF8yePZu7776b0aNHs3HjRqKiovjyyy/L/P58P8dHH32UDRs2EBcXx+uvvw7Ak08+yapVq9iwYQOnT5/myy+/vJxQ5CYYOTk53HnnnUyZMoV169bx1VdfUd3TWXbdunV8/PHHrF+/nhkzZnDgwIFiYzGFq1lTm46HDAl1JBWrRg2YOFHnuvjiC60aZaUJjYlsa9ZoP/qJE7VvfUXYtQs++0yPVRlkZsKXX0JqauU/4W7USC+YOad/96vaXBdVuivU9OkFn+vaVa8+ZGbq5Ce+evXS5fx5vQrpbdKkssfStm1b+nld9pg5cyZvvfUWWVlZHDhwgM2bN9OlS5d8r4mPj+f6668HoG/fvnz77bd+9z1u3LjL2+zZsweA77777nILRM+ePenatWuB1yUlJREVFcVDDz3EmDFjGDt2LAALFizINw7i5MmTl1sGinLDDTcQExND9+7dARgxYgQA3bt3vxxXWd6ft9atW1/uijVhwgTeeOMNnnrqKRYuXMgf//hHMjIyOHbsGH379r2871xbtmyhRYsW9OnTB4A6Xm3bw4cPp1atWgB06tSJvXv30rRp02LjMf7Fx4Pn66/0YmP1qubnn2uJxTp1ImOWWWNMfllZWgkod6bo3C6PFaF9e2jVSifZ695dqw9FsuXLddzBrbdW/sQiV1aWDsJfuVKrBPq53lopWYtFmEhISLh8f/v27fzlL39h0aJFrF+/ntGjR5ORkVHgNdWqVbt8Pzo6+nJ3Il+5V929t3EluGwaGxtLSkoKt9xyC7Nnz2bMmDGXX7ty5UpSU1NJTU1l//79xMfHF7u/3DiioqLyxR4VFeU39uLe3549ey4Pfp86dSoA4vMXS0Q4f/48jz32GJ9++inr16/npz/9qd/P0zlX4PW+sRcWiymZnBz45BPYuzfUkQRXVBSMHavlaEtZX8AYEwZSU2HKFDhzRn+fKzKpAD35HjVKW0ZKcE0trJ09q++hUydNlqqK2Fi49169YP3VV9piUxVarKt0i0VRLQyxsUWvr1GjfC0URTlz5gy1atWidu3aHDx4kHnz5jF69OiAHuOqq67iww8/5Oqrr2bDhg1+u1qlp6eTkZHB2LFjGTBgwOUWk+HDhzNlyhR+/vOfA5CamkqvXr2oVasW6UEs3t+qVat840J27NjB7t27WbVqFf369WPmzJlcddVVXLhwgaioKOrXr096ejqzZ8/mnnvuAcgXc9euXfnxxx9Zs2YNffr04cyZM/kSPlN+qamwfj107BjqSIJPJC+puHBBWzBGjQJPQ5gxJgw5pyfFixbpHBXBnPiySROd62H5cm0hSUwM3rED6euv9ep9VWml9hYTo9WiatbU7zEjI7Ingy0Ja7EIQ3369KFLly5069aNhx56iMGDBwf8GI8//jj79++nR48e/O///i/dunXL1/UHdIzDmDFj6NmzJ0OHDuVPf/oToOVely5dSo8ePejSpQtvvvkmADfffDMffvghvXv3LjB4O1i6du3Km2++SY8ePTh37hyTJ0+mXr16TJw4kW7dunHrrbcyYMCAy9vff//9PPjgg/Tq1QsRYebMmfzbv/0bPXv2ZOTIkVy8eDEk76MyunhR/zlfcQX49Oqrco4ehR9+0AGgx46FOhpjjD85OXoBYNEi6NGjfHNUlNXQodCiRWSXre7WTav/1asX6khCQwRGj9bPoCr875OSdImJRMnJyS4lJSXfc1u2bKGz1XwEICsri6ysLOLi4ti+fTsjR45k+/btxMRU6UascrOfscItWgTffAMPPgjNm4c6mtA7cEDHceXk6KR6V1wR6oiMMd4WL9blqqtg2LCqMzbAVLyNG/VvfqDKFAeDiKx2zhU7QtDOIquos2fPMmzYMLKysnDO8frrr1tSYSrMmTOwbJkOQrSkQjVtCg88ADNmwDvvaHLRpk2oozLG5Bo4EJKStLUi1M6d0xm5Bw+OnARnxw5dhg4FryGTVV5uV9jYWG0Fa9Qo1BEFlnWFqqLq1q3L6tWrWbduHevXr2dkZZylzISNhATtXztsWKgjCS9JSZpcdOxYuefzMCZSnDwJc+bomIC4uPBIKgC2b4cFC/RKdyTIyYF587TLZ3R0qKMJL/HxOkbXOXj7bfBTFDOiWWJhjKlw0dEwYEDFV1KJRAkJ8JOf6CDunBxYt65qVA4xJtwcPKjjnrZtg+PHQx1Nfj176mDuBQsiY7zFmjU6lmzECEss/GnUSLsF16oF770HmzaFOqLAqXKJRWUdU2JCz362CnIOPvoocq6yhdrGjfDpp/CPf1SuGXeNCXc7d+rV4+honTk53LqniOjg39OndfbqcJaRoZWgWrasuAkEK4M6dfRnrVkznc29sqhSiUVcXBzHjx+3E0ATcM45jh8/Tlykz2IUYFu36pUYP9OGGD+6d4frrtOyvDNnwqVLoY7ImMpv82YtpJCYqFeRw7VbYuvW2m3y2291bohw9d13OiZk1KjIGQ8SKvHxOoP7VVfp4+PHI7/FukqN1m3evDlpaWkcPXo01KGYSiguLo7mNjL5sqwsmD8fGjYEz2TmphgicO212jz+2WcwfboO6q5ZM9SRGVN5NWwInTvDjTeG/wzXI0ceycwAAAAgAElEQVRqa0A4n3z27q1/w5o2DXUkkSG3q1h6Orz5JrRrp3NdRGo9naCGLSKjgb8A0cBU59yLPutbAtOABsAJYIJzLs2z7g/AGLSVZT7wpCtl00NsbCytW7cu9/swxhRv1Spt3p0wQWeqNSXXp4/+Y54zRytqWWJhTGDl5GhrarduUL++jnOKBPXq6YRr4axevao7Z0V51KwJV1+tF+TOnoU77wz/RNefoP27F5FoYApwPdAFuEtEfKcKeQl41znXA3geeMHz2iuBwUAPoBvQD7g2SKEbY0opIwOWLNErL+3ahTqayNS+PTz1VN5VvzNnQhuPMZVFZqaO/Zo9G3bvDnU0ZXP0qLZchJN9++CDD8K7m1Y4E9FywuPG6Wf59tuR+Xc/mNcR+wM7nHO7nHOXgA+Am3226QIs9Nz/2mu9A+KAakB1IBY4XOERG2PKpHp1/eM4alSoI4lssbF6u24dvPKKVqsxxpTd+fPw7rs6/uv66yN37pgdO/TizY4doY5EOaflZffvtzkryit3lvdTp2Dp0lBHU3rB7ArVDNjn9TgNGOCzzTpgPNpd6laglojUc859LyJfAwcBAV51zm0JQszGmFJyTq+8dOgQ6kgqj3btdEDpBx/ATTdpH2ZjIkFWlv5NiI3VYgT+TpTatNEKQhcuwPLlBdd36KCVc9LTISWl4PrOnaFxYz0RW7u24Ppu3fT359gxHaSdnq5dn7r49pmIIP37a3fTefP08wt1d9NNmyAtDW6+2RKLQGjTRgsJJCaGOpLSC2Zi4a82gO8YiaeBV0VkEvANsB/IEpF2QGcgd2TsfBG5xjn3Tb4DiEwGJgO0aNEigKEbY0pq9mz9J59b5cKUX0KCTqj0wQc6Y2vr1jYniAl/GRkwa5a2YN5xh3ZB+uabgtvFxuYlFv7W16ypicW5c/7XJyXp35zTp/2vb9JEE4uTJzWGe+/V40Wy6GidI2LWLJ0zIjk5dLFkZen8Go0b63wbJjDCtTpZcSRYpVdFZBDwW+fcKM/jZwGccy8Usn1NYKtzrrmI/AKIc879zrPu10CGc+4PhR0vOTnZpfi7tGGMqTC7d8M778Dw4ZZYVIQzZ7RLVMeO4T+A01RtZ85o68DRo3oV2044A885rRx37Bg88YQmcKGwbBl89RXcd1/kdi0zxROR1c65YlPYYLZYrALai0hrtCXiTuBu7w1EpD5wwjmXAzyLVogC2As8JCIvoC0f1wJ/Dlbgxpji5eRos3zdujBwYKijqZxq19arlDVqhDoSYwp39CjMmKEtEPfcA23bhjqiyklEx7Ft3hzaOHr31qTGkgoDQUwsnHNZIvIYMA8tNzvNObdJRJ4HUpxzc4EhwAsi4tCuUI96Xv4xMBTYgHaf+tI5989gxW6MKd66dXDokF5Jj9T625Ggf/9QR2BM4XJytHtOdjbcf792QzIVp2nT0M4X4ZxO8ta3b+hiMOElaF2hgs26QhkTPNnZ8Je/6BX1Bx6w2VYrWna2DnJNStKBq8aEkwMHtFXNxgEFz86d8OOPMHRo8I555Aj84x86mVukjgcwJReOXaGMMZVUdLR2ecitCGUqlghs2KCDY9u3txYiE3orV+rg6uuusxmXQ2HvXh243qEDNG9e/PaBMH8+HD+uxSWMyWXz4RpjyiUnR28bNdKqIKbiRUXByJFaXtNfeU5jgsU5PcH84gs4fDjv74EJrsGDtXrWvHn6nVS0HTtg+3a45hob82Xys8TCGFMuc+dqc3gl7VUZttq00epQ336rV4qNCbbsbPj0U52bol8/uP320M+nUFVVqwbDhumMzRU9mDsnR6tAJSbamC9TkP0JMMaU2YEDkJqqV6ysC1TwjRihdfkXLw51JKaqcU7nVVm/Xk9ob7jBkopQ69lTW43nz9e5JSrKhg06vmLECOuGaQqyHwljTJk4p1etatSAq68OdTRVU/36Wm7S+rSbYBPRGa27doVevUIdjQFN7EaP1nktKjLJ69ZNx9VZ4QjjjyUWxpgy2bYN9uyBMWMgLi7U0VRdAwaEOgJTlRw9CidOaDc8m/Qu/LRqpUtFycnRpKJbN/TqUmaW9onLzNRmksKWkq7Pzq644MtCRLM0f7clue/vNaXZ1vt+bCy0bh3qT6RYllgYY0rNOViyREsMWv3y0Lt4Ubs/dO5sk5GZirN3L8ycqf3527a1bjDhbM0aLe4wdCj6BzsjA9LT4ezZvMX78aVLxZ70nzoXyzvrenFzq3W0qnEk/JKAyi4xEV58MdRRFMv+LBhjSk1Ey8uePWv9qsNBdLTWsd+7Fx55xL4TE3hbtsDs2VCnDkyYYElFSGVlacWGIhKFQyvrs2pHIt0/WUKDnMMBSQIW7upN+rkoEuWkJRWhECEDGe1PgzGmVLKy9ES2Zk1dTOjFxOhAyg8/hLVrrRXJBNbKlfCvf0GzZnD33VZeNKByWxN8WxCKenzhQrG7HZITy/rzQ/lqc3PuaX+g3GGmna3LhhPNuKbJdupUyyj3/kwZRMgVI0ssjDGlsmABHDwI992nCYYJD507Q4sWsGiR9n+uXj3UEZnK4uRJnXjtttu0m3dYyszUk+4zZ/JOwH3vnz0L58+HOtI8Fy9qTBVw9b9GTCbXNN7OV2ld2HmmPm1rHyvzvpyDeWldqRl7kcGNd+ZfGR2tVzZiYvSHIyZGn8u9773ExuZt72997hJOcnL0A3Au/33vx763Jblf3Gv8bVOnTqg/jRIJs2/QGBPOjh/Xq5e9e1tSEW5EtELUm2/Cd99pCVBjyio7W8/FExN1MkbngnzBNCsr/1X63MQgPd3//Qy7iu6rf8M9rDrainn7uvJIr+VE1a6Z19RcsybUqpV3v3r1ggmCZ9m9vzr7Zidw05hsqvcdlT+BiJDuOSZ4LLEwxpTY/Pn6/+S660IdifGnWTOt0tW+fagjMZHs4kXtVnfkCDz2mJ5zlvv8MScnb1xAYUmC9xJOLQvhRgQSEvInBr6JQs2axNSqxdijtXHxNYjqcneZD9e6OdzbwFOQKDJ645gQssTCGFMie/bA1q16JdzGVoSvfv1CHYGJZGfPwvvvw+HDcOONXl3qnNPKQefPax//Cxf0fu5j71vf586d0/vOhfS9ha1q1fwmBoU+rlGjxM1HbVuUL7SsLL2YZNXmTElZYmGMKZHly7WL58CBoY7EFOfMGZg7F4YMgebNQx2NCQu+iYGfBODYoSxmLGzC+XOOu7psov2/DsJsr+1yckL9LgoXFaUn3bVr552A166dd0Keez8hIXy678TEaGzVqlXoYZyDhQu1B9O115b8defOwWuvaVe47t0rLj5TuVhiYYwpkfHjdRBn2A7eNJdVrw6HDsG8efDTn4bPeZQppZwcTQYyMrR/kvetv+d8b71bDUqQGCze1YfM9EwmtVtB0+On4XiQ3qc/3t19SrLUqGE/6IUQ0YsNmzfrLOklHQO8ZIkmF40bV2x8pnKxxMIYU6TMzLxJPxs2DHU0piSqV9eJsebO1ZOJrl1DHVEVkZOjJ/ClTQAK2/bSpeCE7YQocdzYcj3ns6qRWL2CxjfUqFHyRCEhIWLKa0aCYcP0b8HChTBuXPHbHz0KKSlaurpBg4qPz1QellgYY4q0bJnOjfDIIxAXF+poTEn16gUrVmh54I4dw6+KY8TJydHLvidP+l9OnIDTp8O7u5AfKUdbsu54c+5tv5zq0VlUj84qfOPYWIiP1wShRo28+/HxhT+fe5uQYD+EIVSnDlx5JXzzDQwYoIUeijJ/vn7dQ4YEJTxTidhvuTGmUOnpWrq0fXtLKiJNVJT2jX7vPb3yaGNjilBc0nDyJJw6FXFJQwGxsZdP/l1cPF/v78A3WS3p0O0CXFsb6sQXTAq8EwfrBxnRBg+GNWs0aZg0qfDtjh2D7dth+HDNB40pDUssjDGFWrRIz6WGDw91JKYs2raFm2+GLl1CHUkI5SYNp05pq0KkJQ3Vq+sSF1f627i4/MmBp8UgOxv++U9IzYA+I2DsWOt1VBVUrw633FL8GIv69eHhh/XWmNKyxMIY49ehQ5CaCoMGQVJSqKMxZdW7t946V4nGtvoOaj5/PnxaGuLj807q/Z3olyY5CMgEEgX961/6uz1kiFYJqjQ/F6ZY7drl3ff3NyEjQ3/8bMC2KStLLIwxfqWm6jnSNdeEOhJTXocPwyefwG23hWggZlZW2aoaFfaaIA1qLiAhQaeiTkqCunX1NjExb6lbt8JLhwbClVdqGeJevUIdiQmFrCz49FP9GRg0KO/5S5dgyhTo3x+uvjp08ZnIZomFMcavUaP0H4yNrYh8NWvqhfv58+HuskzA65y2Cpw+rTs6dUrv51ZAKi4pyM4O+HsKON+kwTeBiJCkoTDHj2sRhmHD9O1YK2TVFROjScSSJdCzp/aUA1i6VMfVtW4d2vhMZAtqYiEio4G/ANHAVOfciz7rWwLTgAbACWCCcy7Ns64FMBW4AnDADc65PcGL3piqITtbzwcTEuzko7JISNCWp/nzYdcuaNPGa+XFi3mJwsmTecmDdxJx6pRe5oxUuUmD91KJkobipKXB3/+u95OT9e2aqm3kSPjb3zS5uP56/XVftkwnwrNJNU15BC2xEJFoYAowAkgDVonIXOfcZq/NXgLedc69IyJDgReAez3r3gX+2zk3X0RqAmE60s6YyLZ6tdY6f+QRPf8yESorK19yMODcaVL21mPeb87xcK8VRJ32JBEZGaGOtGy8BzXHxemIVN/kIXepxElDcbZtg48/1qkhJkywpMKoBg10jopVq6BfP/j2W22YHDYs1JGZSBfMFov+wA7n3C4AEfkAuBnwTiy6AD/33P8amOPZtgsQ45ybD+CcOxusoI2pSi5cgMWLoWlTOwEJW7lVjnxbFHwfnzuX72UxwPD0Jny0qy+bM87QLelw8GKOiirb4OXCBj5Xq2Yjjktg7VqdJLFJE7jnHisdavIbMgTWr4cvvoD9+3W8hf3dN+UVzMSiGbDP63EaMMBnm3XAeLS71K1ALRGpB3QATonIJ0BrYAHwjHMuAjruGhM5vv1Wk4tRo+y8LSRyuyXlVjPyvZ+bQDhXpt13STzI7W1X06nuodK/uHp1PevIbRmoU0fPVEuSFFTgxGhnzuhcK97juaOj4cYb9f7q1bBvX/7XVK+u3T9AJxE8eDD/+oQEGDFC7y9dqrMQe6tTB667Tu8vWaJfkbd69fIGvy5cqP3WvTVqlDdodt48/Z3z1qyZXkUG+PxzyMzMv75FC+jTR+//4x8FfxzatIEePTTOjh11puUq3GhjCpGQALffrhWgnLNpSkxgBDOx8Hea4vvf8WngVRGZBHwD7Aey0DivBnoDe4FZwCTgrXwHEJkMTAZo0aJF4CI3pgo4cUJPsnr1slKDAeccnD1bdNJw8mTBM8wAE9HkAiDHCVHi9KS/Tp2CSYPv4zAdxb9ihU765X013juPOXoUdu/O/5rcwaoAR44UXO9d5//QIdi7N/9678paBw7oNt4uXsy7n5amv1vevBOBffsKJh7eJ3g//ph/f77x79lTsJpu7lXnNm18xtMY46Nt21BHYCobcWW88lXqA4kMAn7rnBvlefwsgHPuhUK2rwlsdc41F5GBwIvOuSGedfcCA51zjxZ2vOTkZJeSkhLgd2FM5bVihV5dffxx7Y9tSsh7LENuouDbyhCKwc8ieoacmyTkLnXqsPNMA+Yub8hPH46lTpMaEdk8lZOjPayys/Xjtsm8jDGm4ojIaudccnHbBbPFYhXQXkRaoy0RdwL5Ch+KSH3ghHMuB3gWrRCV+9pEEWngnDsKDAUsazAmgAYMgK5dtTSp8XBOLzcfOeI/aTh5suDl5mCoWTOvRcEnabh8v1atQqdTrncKzq2Ghcu1m0yk2bxZxwJNnKgtFZZUGGNMeAhaYuGcyxKRx4B5aLnZac65TSLyPJDinJsLDAFeEBGHdoV61PPabBF5GlgoIgKsBt4MVuzGVGbOaY37+vWreFKRnq79Wg4c0JGM+/fr/WBWTYqOzksMcidc871ft265xyzUrQsDB+rYhAEDtE9/pFixAr78UktiRmBDizHGVGpB6woVbNYVypiSWb9eZ2F94IEqUr/84kUdrZubPOQmEGfOVOxx4+LyEoTCkoZatYJ2tnzxIrz8siaUkyaF/0m6c7BggQ6m7tQJxo+3wabGGBMs4dgVyhgTZjIzdVxFkyaRddW6RLKytAuTd/Kwfz8cOxbY44hoQlBc0hBmg5+rV9fKRp99poOTW7YMdURF+/ZbTSr69dOKToX08jLGGBNCllgYU4UtX67jjseNC/8r1oXK7cvlnTzs3w+HD+vI3vKKi9OJPerV85801KmjXZgiUJ8+2mIR7kkF6IzR8fF6G7E/q8YYU8lZYmFMFXX2rF4F7tw5Mk4sAe2u5J087N+v3Zp863GWRUyMNt00barNN7lL3bqV9kw2KgpatdL7mZnh17UoPV1/RkeN0hKruXM7GGOMCU+WWBhTRaWl6fny8OGhjsSHczpr9OHDeUlE7qDqQFRgEtGJCLyTh6ZNoWHDKtu/Zu1aHb/w6KP550gIpaNHYcYMndqjTx+bW8UYYyKBJRbGVFGdOsG//7v2tQ8657Rc65EjegaZu+Q+DlQlprp18ycPzZppq0S4XZoPsWbN4Px5nUU6d0bqUNq7F2bO1B5m999vSYUxxkQKSyyMqWKc09aK5s0rOKnIytKxD96JQ27ycOxYYCeMq1Ejf/KQexsul9/DXMOG0LcvrFql3Y1COS/EDz/Ahx/q0JUJE3QoizHGmMhgiYUxVcyOHfD++3D77dClSzl3dvGi/xaHo0d1YrlAl7OOjc0rYeU9FqJOnUo7DiJYrrsONmyA+fPhrrtCF0edOjruY9w4ywuNMSbSWGJhTBWSkwNffaUFjjp2LMELcsc7+HZZyl0qau6H2FgdB9G4cf6xEPXrV9lxEBUtIQGuvhoWLdKcMCkpeMd2DrZvh/btoVEjbakwxhgTeSyxMKYKWb1a84G77vKpkJqTA4cOwZ49Omi6IsY7+EpI0OTBd2nYEGrXthaIEBg4UMfeBDOpyM6GuXNh3Tq4915o2zZ4xzbGGBNYllgYU0VkZMDXX2s3kw6NTkPqbtjtWX78sWISiLp1CyYNufetn0vYiYnJG19x/nzFf0UXL8KsWbBrFwwdCm3aVOzxjDHGVCxLLIyp7C5ehB9/5Niqg0SvrMaoxouQ7/cGZt9RUdqvyjdpaNBAz1CrVQvMcUxQff01rFkDjz9ecV9herqO9TlyBG65BXr1qpjjGGOMCR5LLIypTHJydMK43bu1W9Pu3ToPhHM0B55KFKIzSzmgOne8g79Wh6SkiJ112hSuXTstPbtsGQwZUjHHOHAATp6Eu+/W4xljjIl8llgYE8lOnsxLIHK7NPmZhXrH6Qa0qX2M6KgikoqEBO0n1aJF/uTBKi5VOVdcAV27wtKlOjld7dqB2/eFCxAfr8UDnnzSesQZY0xlYomFMZEiI0MTB+9E4tSpYl+272wiM7YPYPQVmxjYaLc+GROjZ4+tWkHr1ro0aGAJhLls+HDYulWrRN1yS2D2uXUrfPop3Hmn/shZUmGMMZWLJRbGhKOcHO0rsttrgPXBg6WeF8I5mJfWhVpJMfS5oTG0H6BndM2ba3JhTCESE7VK1Jo1ea0M5bFqFXzxhVYNbtgwMDEaY4wJL3ZmYUyoOee/S9OlS2XbX82amjy0asWmrE6k1W/BLbdXo5oNjjWldM01MHhw+ZIK57TV49tvoUMH+MlPdNiOMcaYyscSC2OCKSdH54nYt0+XtDS9TU8v2/5iYnRMRG53ptattUqTCJmZMP9VaNwCevYM7NswVUP16nrrnP6IlmWsxdatmlT07Qtjxtj8hsYYU5lZYmFMRcnI0IpM3knE/v2QmVn2fTZufLk1gtattV9JIV2a0tMhLg5GjbKhE6Z8Pv1Uf3R/9rPSFwHr1EknZOzQwX4OjTGmsrPEwpjyck4HUXu3QKSlaYH+8qhVK39LRMuWpRrtmpQEjzxiJ3Om/Lp3h/XrISUFBgwofvuzZ2HOHLj+em1A69ix4mM0xhgTepZYGFMa2dlw6FD+JGLfPjh3rnz7jY3VxMG7SlNSUpmzgs2bdRbjuLjyhWUM6DwTbdvC4sXQo0fRYy6OHYMZM/RX4tQpTSyMMcZUDZZYGFOYCxc0efBOIA4cgKys8u23dm0t9dq8ud5ecYWWyQlQ5/MjR+Cjj+DKK2HEiIDs0lRxIjByJLz2GnzzjXav82ffPpg5U7efNEl76hljjKk6LLEwJrcqk++A6mPHyrdfEWjUKC95yE0kAjnbmB9ffaWDbgcPrtDDmCqmUSOdLG/HDp3jwnesxb598M47+uM9YYI2uBljjKlagppYiMho4C9ANDDVOfeiz/qWwDSgAXACmOCcS/NaXxvYAnzqnHssaIGbyiUnR0vVbNyYl0icP1++fVarlr8FonlzaNo0r6xOkOzYocuoUTb5mAm8ESO0VoC/AdyNG0Pv3jBkiE7ibowxpuoJWmIhItHAFGAEkAasEpG5zrnNXpu9BLzrnHtHRIYCLwD3eq3/HbAkWDGbSubwYfj+e11KMGN1oerWzUsicm8bNAh5Hc2cHG2tSEqCfv1CGoqppHLH7Fy6pGMo6tbVie969NB1Y8aENj5jjDGhFcwWi/7ADufcLgAR+QC4GfBOLLoAP/fc/xqYk7tCRPoCjYAvgeRgBGwqgYwMWL0ali6FnTtL99qoKL0M651ANG+u1ZrCUEaGnuj17m2TapuK9d57OtSoUSNITdWkduDAUEdljDEm1IJ5+tEM2Of1OA3wLVy4DhiPdpe6FaglIvWAk8D/oq0Xwyo+VBPRnIMffoBly2DNmpLNYB0Xp0mDd3empk0jaorgGjXg7rv17RtTkfr1g08+gYMHtetTSUrQGmOMqfyCmVj4q5vpewr0NPCqiEwCvgH2A1nAz4AvnHP7pIjymyIyGZgM0KJFiwCEHNmc0xlve/bUAZXnz1fyvs/Hj+d1dSpu4HW1ajoVcI8emkTUrx/REz5s2KAVeMpRodaYEuveXSfMa9rUZnU3xhiTJ5iJRRpwhdfj5sAB7w2ccweAcQAiUhMY75w7LSKDgKtF5GdATaCaiJx1zj3j8/o3gDcAkpOTq/R126ws+Mc/9IQzKkqHFxw5Ag8/HPKhAIF16RKsXautE1u3Fr99+/Zah7VPn0ozycOpU/pdd+sGt9wS6mhMVSCik98ZY4wx3kqcWIjIn9FKThvLeKxVQHsRaY22RNwJ3O1zjPrACedcDvAsWiEK59w9XttMApJ9kwqTJyMDZs2C3bu1LOTgwTph2oYN2h+6T59QR1hOzsGuXZpMpKToGy5KYiIMGqRLw4bBiTGIFi7UE73rrgt1JMYYY4ypykrTYtEPeFxEVgNTgQ+cc2dK+mLnXJaIPAbMQ8vNTnPObRKR54EU59xcYAjwgog4tCvUo6WIzwDp6Trr7dGjcOuted0UunTRHj+LFumV7WrVQhtnmZw6BcuXa0Jx+HDR28bE6CjmwYOhY8dK1kyTJy1NE8ZrroE6dUIdjTHGGGOqMnGlGOkpIh2BnwITgDrAJ8BbzrmwKwGbnJzsUlJSQh1G0F28CH//u55otm2bf11aGkydCtdeG0FXtzMzYd06TSY2by5+ZHKrVppMJCdX+okcnINp03RuvyeeiNBk0RhjjDFhT0RWO+eKrcpaqjEWzrltwC9F5FngBjTJ+EpE9gJvAW84506UJWBTPvv3ay+f6tVh0iT/A3ibN9fWik2bNLkI24v4zsHevZpMrFxZ/OR1tWtrrctBg3Q0aRWRna1vt29fSyqMMcYYE3plHbwdC9RGWy2igb1oKdjnRGSyc+7vAYrPlMDmzVr6sW9fHVBZVFWgG27QCqphmVSkp8OKFTrnxIEDRW8bHa0Vna68UrOlsHxDFSsmxgbQGmOMMSZ8lCqxEJFktJXiTuA88A7woHNut2f9k8D/AyyxCJIVK+DLL7U14tpri98+t3dQVpY2BNSuXbHxFSs7GzZu1GRiwwadaasozZtrMjFgANSsGZwYw9CGDfrdtWwZ6kiMMcYYY1RpqkJtADqig68nAZ8757J9Nvs7mliYCuYcLFig5+OdOsH48SWfy805ePttveJdWLepCrd/v3Z1WrFCWyqKkpCgicSVV+oI9Cru3Dn47DNNKiyxMMYYY0y4KE2LxYdoJaf9hW3gnDsKVL0+KSFw9qyWju3XT7vDlKYnkIgWTPrsM536oXPnioszn1OnNOhly+DHH4veNioKunbVZKJHD82CDACLF+uY9hEjQh2JMcYYY0ye0pyt/V/8JA0iEgfkOOcuBSwqU6hLl7RlolYtneyuVq2ytTj06aPjoufPhw4ddMhCwOUOwl6/XvvuFJdMADRpktfVyeqnFnD0KKxerUWvGjQIdTTGGGOMMXlKk1h8BCwB/uTz/CPo/BM2528FS0+H99/XaRmuu6584yOiomDkSJ3zYuVKLagUEBcvajNIbjJx+nTxr4mP16aXK6/UcrEh6ZsVGb76SitAlWQ8jTHGGGNMMJUmsRgM/MrP8/OB/xOYcExhjh3TJOD8eWjRIjD7bNdOl127tFprmc/nT5zISyS2btWR4cUR0cEhV16p/bJKOkCkCnMO2rTRFqaEhFBHY4wxxhiTX2kSixqAvzPGHKBWYMIx/uzdCzNnanel++/X3kKBctttOvdFqZKKnBzYsycvmUhLK9nrRLRFomdP7eqUlFSGiKsukQC2LBljjDHGBFhpEov1wF3Ab3yevxvYGLCITD4XLmj3p5o1YcIESEwM7P7j4vT27Fmt/FrosIaMDJ0wY8MGXYqr5JSrenXo0kUHYHfrFgb1bSPTxo36/fToYT3FjDHGGBOeSpNY/A6YIyLtgEWe54YBPwFuDXRgRsXHaynZ5s3z5kPJgXcAACAASURBVKAItOxseOMNbQm56y6vFceOaavE+vXwww+6YUnUq6dnwD16aL8dq+hULhcv6lwlSUn6kRpjjDHGhKMSn/E55z4XkRuB54CXPU+vBW5yzv2rIoKrqpyDRYugcWOtuNqhQ8UeLzoa+veHBfMduxftofWptZpMHDxYsh2IQNu2etbbvbtmKHZZPWCWLtUWpTvvtI/VGGOMMeGrVJeSnXNfAl9WUCwGbRSYOxfWrdNhCF27VvABz5+HTZsYuGUjKUsSmbfoHJM7f0uUuKJfFx+vwfXoobdVeBbsinT6tE770b27tloZY4wxxoQr66MSRi5ehA8/hJ07YehQuPrqCjrQ4cN5XZx27ICcHGKA4Q2a8vGuPqw/3oxe9f0MyG7YMK+LU7t2FTT5hfG2cKHeDh8e2jiMMcYYY4pT4sRCRKqh5WbvAloA+eqDOufsLLMcLl2C6dP1nP/mm7UCa8BkZ8P27Troev16OHLE72ZdEw+wvGZrDl2oA6TpZBft2+d1cWrUKIBBmZLo0gWaNbO5Ao0xxhgT/ko7ePsO4AXg/wG/AFoBdwL/FfDIqphq1fQcfuhQvQ2Y1avhgw/gzJliNxWBSb3XE9OjC/R4SM9qK2rEuCmRTp1CHYExxhhjTMmUJrG4HXjEOfeliLwE/MM5t1NEtgAjgNcrJMJKbt8+nRuucWNNKgImKwtmz9ZR4MVp0uRyF6eYNm0gKorDh6FGtk1QEipbt8KhQ3DVVVZUyxhjjDGRoTSnLI2AzZ77Z4G6nvtfAv83kEFVFVu3wscfa1eXSZMCWPHnxAmtH7t7t//10dFaaqpnT+3iVL9+vtUXLsDUqTom+5ZbAhSTKbGsLJg3T1uxrrkm1NEYY4wxxpRMaRKLvUBTz+0OYBSwGhgEXAh8aJVbSgp8/jk0bQq33x7ApGLTJnjrLTh3Lv/zMTFaU7ZHD+jcOW9mPD/i43XTZcu0MlUgZ/o2xVu5Ek6ehHvv1WEuxhhjjDGRoDSJxafohHjLgb8AM0XkIaAZ8McKiK1Scg6+/hq++UYbDW67Ta9Ml1tOjmYqn3+uB/FWrx5MngytWpV4d1dfDWvX6pXziRNt/oRgOX9efzbat9epQYwxxhhjIkVpJsh71uv+xyKyDxgM/OCc+6wigquMcnJg/37o2xfGjAnQFen0dG2l2LKl4Lru3eH++yEhoVS7jIuD667TPGXbNhtEHCyLF2uFsJEjQx2JMcYYY0zplCixEJFYYAbwf5xzOwGccyuAFRUYW6Vy8aJWfa1RA+66S4c5BKQVYOdOHU9x6lT+50W0bu3o0WU+UN++sGaNTtJmgqN3b21gatAg1JEYY4wxxpROia6XO+cygZFAMdMxF01ERovINhHZISLP+FnfUkQWish6EVksIs09z/cSke9FZJNn3R3liSPYzp7VOSpmztReSjExAUgqnIMFC+CllwomFbVrw89/DtdfX64DRUXBQw/pOAsTHE2a2OdtjDHGmMhUmo44nwDjynogEYkGpgDXA12Au0Ski89mLwHvOud6AM+jc2YAnAfuc851BUYDfxaRukSA48e1l9KxY1rhJyCtFBcuwOuvw0cfad8qb+3bw3PPQceOATiQJhfO6fx6F2yIfoXZtQvmzLHP2BhjjDGRq7RVoZ4TkauBFCBf2SHn3J+KeX1/YIdzbheAiHwA3ExeCVvQhOPnnvtfA3M8+/7B6zgHROQI0ADwuVQfXtLS4O9/90w8N0nLypbbvn3a9cnf7NmjRml92ACXEjp5Ut/HgAHas8oEVk4OfPUVZGTonCbGGGOMMZGoNInFJOAk0MOzeHNAcYlFM2Cf1+M0wLfTxzpgPFp16laglojUc84dz91ARPoD1YCdpYg96JyDzz7TQdATJkBSUgB2unSp9qfKzMz/fHy8DtDu2TMABykoKQn69NEyqP366RgAEzgbN+pkeLfdZpPhGWOMMSZylaYqVOtyHstfJyDfMRtPA6+KyCTgG2A/kHV5ByJNgPeAic45nz5AICKTgckALVq0KGe45SMCd9yhpWRLWZCpoEuXNKFYtqzguhYt4OGHC0xyF2jXXQcbNuiwjjsiaoRL+Nu0CRITdUJCY4wxxphIFczro2nAFV6PmwMHvDdwzh3AM45DRGoC451zpz2PawOfA88555b7O4Bz7g3gDYDk5ORyDTQPhMTEAOzkyBEdT5GWVnDdNdfo7HpB6D9TsyZcdRUsWgR79pRqSgxThOxsnSC9Z0+bK8QYY4wxka3EiYWIvFzUeufcE8XsYhXQXkRaoy0RdwJ3+xyjPnDC0xrxLDDN83w1dIK+d51zH5U05oi3Zg288452vvdWrRrccw8MHBjUcAYN0jktfMMxZZeZqd3L2rcPdSTGGGOMMeVTmhaL7j6PY4FOnn2sKe7FzrksEXkMmAdEA9Occ5tE5HkgxTk3FxgCvCAiDu0K9ajn5bcD1wD1PN2kACY551JLEX/kyM6GTz7Rfke+GjWCRx6Bpk2DHlZsLDz4oF1ZD6S4OBgxItRRGGOMMcaUnzhX9h5DIhIHvAV865x7LWBRBUBycrJLSUkJdRild/IkvPmmTnznKzkZ7r1Xz0ZDKDsbUlOhRw+rYlReaWk6d0V0dKgjMcYYY/7/9u49zq6yPPT478mNIAkhgQAhFwh3kEukIxRBuSk3LZSKFUSLpxxRW+2xYr20tLVUCsdibTmVVo61oC1YakulLdciVqlACSEJd9gJgQxBCIRLAgkhydM/1hrZTGaSmezZe83e8/t+PvPZe6/1rrWeeVnZrGfem9S3iLgnM7s2V66heUkzcw1wIfB7jZxHpYcegq98ZeOkYvRoOOOMormg4qQC4Omn4V//te+x5Bq4lSvhW9+CO/scMSRJktRehmLw9lRgwhCcZ+TasAGuv76Yn7Z3C9KUKXDuuTC70Um5hs6MGbD//nD77cU0tBMnVh1Re6rVitc996w2DkmSpKEwmMHbn+29CZgGnAVcP5RBjSirVhVLcz/44Mb7DjgAfv3Xh2C+2qH37ncXA7l/+EM49dSqo2lPtVqRlO24Y9WRSJIkNW4wLRaf7vV5A7Ac+FvgoiGLaCRZvLhYRfuFF968PQJOOQVOOmnYjpSeMqVYifuOO4rXnXeuOqL2smFD8Z9/332H7X9iSZKkQWnlAnnqkVn8qf/73y+eMOtNnFiMpdh332piG4R3vasYb7F+fdWRtJ+nnoLVq+0GJUmSOsdgukKNA0aVA7brt48HNmTm2qEOriOtWVOsTTGvjxl699wTPvYx2G671se1BcaPh7PPrjqK9jR9etHLbaedqo5EkiRpaAxmVqh/BH6jj+2fAK4ZmnA63FNPwYUX9p1UHH88fPazbZNU1FuzBn7yE1suBmPUKJg1C7baqupIJEmShsZgEosjgJv72H4L8I6hCaeD3XEHXHQRPPvsm7ePH18sePf+97ftYgZPPgm33gr33FN1JO3h1Vfhxhvh+eerjkSSJGnoDGbw9luAdX1s3wA44Wh/Xn8drr4a/uu/Nt43cyZ8/OMwdWrr4xpCe+1VzIb7ox/BgQfC1ltXHdHwtmhRsXbFAQdUHYkkSdLQGUyLxULgzD62fwi4f2jC6TDPPgsXX9x3UnHkkfCFL7R9UgHFrEYnnFAMRv7JT6qOZvir1Yrka5ddqo5EkiRp6AymxeKPgX+JiD2BH5bbjgM+AJw21IG1vXvvhSuuKAYg1Bs7Fs46Cw4/vJKwmmXnnWHOHLjrLujqKqaj1cYyixaLPfYoxllIkiR1isFMN/vvEfFLwPnApeXme4FTMvOGZgTXltavh2uvhVtu2XjfjjsW4ymmT299XC1w7LHw2muuy7ApzzxTrInoNLOSJKnTDKbFgsy8EbixSbF0hhUr4Mc/3nj7IYcUc7OOH9/6mFpk4kT41V+tOorh7cUX4S1vKVosJEmSOsmAO2NExFERcVQ/2981tGG1salT4dd+7Y3Po0bBBz8I557b0UlFvRUrigabzKojGX723Rd+53eKJEySJKmTDKaX99eByX1s37bcpx5dXXDMMTB5cvEUeeyxI6p/0NKlxXj1++6rOpLhJbP4GUG3giRJGkEGk1jsAyzoY/t95T7VO/10OP982H33qiNpuYMOKmY8+o//KGbbVeGRR+DSS4sWHUmSpE4zmMRiNdDXBJkzgLVDE04HGTMGJkyoOopKRBQLib/8crEuoAq1WrE43qRJVUciSZI09AaTWNwEXBwRP+8OFRFTgD8p90k/t9tusN9+cPvtxSxII11mkVjMnt22C6xLkiRt0mBmhfoc8GNgSUQsLLcdBCwHzhjqwNT+3vMe2HZbH6QBnn++mBHqyCOrjkSSJKk5BrOOxdMRcTBwFjAHCOBK4KrMfLVJ8amNTZkCJ51UdRTDQ61WvLp+hSRJ6lSDWseCYizFA8BKYFy57fSIIDO/M6SRqWM8+SQ8+CCccMLInRFp553hiCNgu+2qjkSSJKk5BpxYRMS+wL8CsylaK9aXx78OvAaYWKhPy5bBnXcWE2TtvXfV0VRjt92KH0mSpE41mMHbfw7cA0wCXgX2A7qA+cD7B3KCiDgxIh6JiFpEfLGP/btGxK0RsTAifhQRM+r2nR0Rj5U/Zw8iblXs7W+H7beHm2+G9eurjqb1XngBnn3WBQMlSVJnG0xi8XbgK5n5CrABGJOZ84DPA1/b3MERMRr4BnASsD9wZkTs36vYJcB3MvMg4ALgovLYKcAfAocBhwJ/WD87lYa30aOLgdzPPQfz5lUdTevddRdcfjmsW1d1JJIkSc0zmMQiKFoqoJgJanr5vhsYyJDUQ4FaZi7OzLXA94BTe5XZH7i1fH9b3f4TgFsyc0VmvgDcApw4iNhVsX32KboC3XYbrFlTdTStVasVv/vYsVVHIkmS1DyDSSzuBw4u3/838IWIOAr4I6A2gOOnA0vrPnfzRnLSYwFvdKs6DZgYEdsP8FgNYxHF4O13vnNkPWC/+GLRUuNsUJIkqdMNJrG4kKLVAuB8YCZFq8LxwG8N4Pi+5gPq3ev8c8BREXEvcBTwFLBugMcSEedGxNyImLt8+fIBhKRWmjYNDj98ZK1r4TSzkiRppBjMOhY31b1fDOxfjn14IXNAw1K7KZKRHjOAZb2usQz4FYCImAC8PzNfiohu4Ohex/6ojxgvBy4H6OrqcqjsMLRuHSxYADvuCDNnbr58u1u0qJhidvvtq45EkiSpuQa7jsWbZOaKQRS/G9grImZTtEScAXyovkBE7ACsyMwNwJeAb5e7bgL+pG7A9vHlfrWZCLj11mLq2ZGQWJx6atEdaqSu3yFJkkaOwXSFakhmrgM+RZEkPARck5kPRMQFEXFKWexo4JGIeBTYiaL7VU8C88cUycndwAWDTGo0TIweDQceCA8/DKtXVx1N840fXyyOJ0mS1OliYL2Y2k9XV1fOnTu36jDUh2XLiulX3/c+6OqqOprmmTevmAHrHe+oOhJJkqQtFxH3ZOZmn9pa1mIh9Zg2rRhjMX9+1ZE013//Nzz6aNVRSJIktYaJhVouAubMKd6vXVttLM2yciX87GfOBiVJkkaOhgZvS1vq8MM7u4vQokXFq4mFJEkaKWyxUCV6ZklavRo2bKg2lmao1WDCBNhpp6ojkSRJag0TC1Wmuxu+9jVYvLjqSIZeJuyzj9PMSpKkkcOuUKrMzjvD2LHFIO5O6zL0gQ8UyYUkSdJIYYuFKjNmzBtrWqxZU3U0Q6ena5etFZIkaSQxsVCl5syBdevggQeqjmToXHEFXHdd1VFIkiS1lomFKtVpa1qsXg1Ll8LEiVVHIkmS1FqOsVClIuC97y1mUOoEixcXYys6bcyIJEnS5phYqHK77lp1BEOnVoOtt4bp06uORJIkqbXsCqVh4amninEJ7bymRWaRWOy+O4zyX5YkSRphfPzRsPDSSzBvHjz+eNWRbLkNG+DII+GQQ6qORJIkqfVMLDQs7L130YWonQdxjx4Nhx0Ge+xRdSSSJEmtZ2KhYaFnTYuHHmrfNS0WLYJXXqk6CkmSpGqYWGjYaOc1Ldauhauugp/+tOpIJEmSqmFioWFj2jTYa6/2HPj8+OOwfr3TzEqSpJHL6WY1bETAWWdVHcWWqdVg3DiYNavqSCRJkqrRhn8bVqdbvx6ee67qKAauZ5rZ2bOLAdySJEkjkYmFhp1rr4XvfKd91rR48UV44QW7QUmSpJHNxELDzn77wcsvt8+aFpMnw2c+AwccUHUkkiRJ1TGx0LCzzz7tt6bFdtsVMUuSJI1ULU0sIuLEiHgkImoR8cU+9s+KiNsi4t6IWBgRJ5fbx0bElRFxX0Q8FBFfamXcaq0xY4q//rfDmhbr1sE//RMsXVp1JJIkSdVqWWIREaOBbwAnAfsDZ0bE/r2KnQ9ck5lvA84ALiu3fwDYKjMPBH4B+HhE7NaKuFWNnjUtHnmk6kg27Ykn4L77hn8CJEmS1GytbLE4FKhl5uLMXAt8Dzi1V5kEti3fTwKW1W3fJiLGAFsDa4GXmx+yqrLLLnDOOXDQQVVHsmm1WtHCsuuuVUciSZJUrVYmFtOB+g4j3eW2el8GPhwR3cD1wKfL7d8HXgGeBp4ELsnMFU2NVpWKgJkzi9fhrFYrkopx46qORJIkqVqtTCz6ekTMXp/PBK7IzBnAycB3I2IURWvHemAXYDZwXkTsvtEFIs6NiLkRMXf58uVDG71aLhNuvhl++tOqI+nbSy/B8uVOMytJkgStTSy6gZl1n2fwRlenHucA1wBk5h3AeGAH4EPAjZn5emY+C/wX0NX7Apl5eWZ2ZWbX1KlTm/ArqJUiigf3O+8cnmtarFwJO+0Ee+xRdSSSJEnVa2VicTewV0TMjohxFIOzr+tV5kngOICI2I8isVhebj82CtsAvwg83LLIVZk5c4o1LZYsqTqSjc2YAZ/8JOy4Y9WRSJIkVa9liUVmrgM+BdwEPEQx+9MDEXFBRJxSFjsP+FhELACuBj6amUkxm9QE4H6KBOVvM3Nhq2JXdfbZB8aPH35rWmzYAOvXVx2FJEnS8DGmlRfLzOspBmXXb/uDuvcPAkf0cdwqiilnNcKMGQMHHlgkFmvWFEnGcNDdDX/3d/CRjxSDzCVJkka6liYW0pZ429uKNS1ef334JBa1WhGTQ3kkSZIKJhYa9nbZBU7tveJJxWq1YozFcEl0JEmSqtbKwdvSFsuEp58uBnJX7ZVXYNkyp5mVJEmqZ2KhtvDKK3D55TB3btWRwKJFxauJhSRJ0htMLNQWJkwo1ouYP7/6NS2mTYNjjileJUmSVDCxUNsYLmtaTJ0KRx1VLOAnSZKkgomF2sa++1a/psWLLxZdoVzDQpIk6c1MLNQ2xoyBAw6AxYure7BfuLBYv2LNmmquL0mSNFw53azayjHHwAknwOjR1Vy/VivGVmyzTTXXlyRJGq5ssVBb2WYbGDu2mmuvWVOsuO1sUJIkSRszsVDb6e6Gyy6D559v7XUXLy5mpDKxkCRJ2piJhdrOttvC8uWwYEFrr7tkSTF4fMaM1l5XkiSpHZhYqO1su22xpsWCBcWK3K1y4olw7rkwyn81kiRJG/ERSW1pzhx46SV4/PHWXXPUKJgypXXXkyRJaicmFmpL++zT2jUt5s+H66+vftVvSZKk4crpZtWWxo6FY4+FCRNac72FC2HVKrtBSZIk9cfEQm3r0ENbc521a+GJJ+Cww1pzPUmSpHbk31/V1laubP7sUEuWFCt9O82sJElS/0ws1NYWLIBrr4UVK5p3jVqt6Ho1a1bzriFJktTuTCzU1g46CCKa22qx1Vbw1rfCGDsOSpIk9ctHJbW1bbeF3XcvZm06+ugiyRhqxx039OeUJEnqNLZYqO31rGmxZMnQn/u111q7CJ8kSVK7amliEREnRsQjEVGLiC/2sX9WRNwWEfdGxMKIOLlu30ERcUdEPBAR90XE+FbGruFr331h663h6aeH/tz//M9w5ZVDf15JkqRO07KuUBExGvgG8B6gG7g7Iq7LzAfrip0PXJOZfxUR+wPXA7tFxBjg74CPZOaCiNgeeL1VsWt4GzsWfvu3Ydy4oT3vunXFyt5z5gzteSVJkjpRK1ssDgVqmbk4M9cC3wNO7VUmgW3L95OAZeX744GFmbkAIDOfz8z1LYhZbaInqVg/hHfF0qXFGhZOMytJkrR5rUwspgNL6z53l9vqfRn4cER0U7RWfLrcvjeQEXFTRMyLiM83O1i1n3/7N/jud4fufI89BqNHw267Dd05JUmSOlUrE4u+5uvpPSz2TOCKzJwBnAx8NyJGUXTZOhI4q3w9LSI2mqsnIs6NiLkRMXf58uVDG72GvUmTigHcQ7WmRa1WrF0x1F2sJEmSOlErE4tuYGbd5xm80dWpxznANQCZeQcwHtihPPY/M/O5zHyVojXjkN4XyMzLM7MrM7umTp3ahF9Bw9nBBw/dmhaZcMwxcMQRjZ9LkiRpJGhlYnE3sFdEzI6IccAZwHW9yjwJHAcQEftRJBbLgZuAgyLiLeVA7qOAB5Hq9KxpsWBB41PERsB++zm+QpIkaaBallhk5jrgUxRJwkMUsz89EBEXRMQpZbHzgI9FxALgauCjWXgB+DOK5GQ+MC8z/71Vsat9zJkDL77Y+JoWDz8Mzz47JCFJkiSNCJEduvpXV1dXzp07t+ow1GKvvw533110i9pmmy07x4YN8NWvFi0Wp/aet0ySJGmEiYh7MrNrc+Vato6F1Apjx8I73tHYOZ56CtassRuUJEnSYLR05W2pFTZsgIULYdGiLTu+VivGWOy++9DGJUmS1MlssVDHiYD//E+YOBH22GPwx9dqMGMGbL310McmSZLUqWyxUMeJKAZxL1kCL7wwuGNfew2eecZuUJIkSYNlYqGOdNBBW7amxVZbwec/D4ce2py4JEmSOpWJhTrSpEnFGIn58we/psW4cXaDkiRJGiwTC3WsOXOKJGHVqoGVz4SrrirWsJAkSdLgOHhbHeuAA4qfiIGV/9nP4NFH4a1vbW5ckiRJncgWC3WsiOJn7VpYt27z5Wu14nVLZpKSJEka6Uws1NFWrIBLLoH779982VoNpk2DCROaH5ckSVKnMbFQR5s8uVjPYv78TZdbswaWLnWaWUmSpC1lYqGOFgEHH7z5NS1Wry5mkdprr5aFJkmS1FFMLNTxDj5482taTJ4MH/4wzJrVurgkSZI6iYmFOt6kSTB7dv9rWmTCK6+0Pi5JkqRO4nSzGhHe/W4YNarvqWefew4uuwxOP92pZiVJkraUiYVGhF126X9frVa0Wkyf3rp4JEmSOo1doTRiLF8O111XrGtRr1aDHXaA7barJi5JkqROYGKhEWP1apg3Dx588I1tr78OTzzhNLOSJEmNMrHQiDFzJkyZ8uY1LZYsKVblNrGQJElqjImFRowImDPnzWtaTJsG730v7LprpaFJkiS1PRMLjSi917SYMAHe/nYYO7bauCRJktqdiYVGlEmT4IADikRi5Uq4915Ys6bqqCRJktpfSxOLiDgxIh6JiFpEfLGP/bMi4raIuDciFkbEyX3sXxURn2td1Oo0738/HHEEPPII/OAHsGpV1RFJkiS1v5YlFhExGvgGcBKwP3BmROzfq9j5wDWZ+TbgDOCyXvu/DtzQ7FjV+TLhjjuKKWa3377qaCRJktpfK1ssDgVqmbk4M9cC3wNO7VUmgW3L95OAZT07IuKXgcXAAy2IVR3uhhvg+edh1qy+V+OWJEnS4LQysZgOLK373F1uq/dl4MMR0Q1cD3waICK2Ab4A/NGmLhAR50bE3IiYu3z58qGKWx1ov/3e/CpJkqTGtDKx6Ovvwtnr85nAFZk5AzgZ+G5EjKJIKL6emZvsDZ+Zl2dmV2Z2TZ06dUiCVmeaPRvOO8/EQpIkaaiMaeG1uoGZdZ9nUNfVqXQOcCJAZt4REeOBHYDDgNMj4qvAdsCGiFiTmX/Z/LDVqSZOrDoCSZKkztHKxOJuYK+ImA08RTE4+0O9yjwJHAdcERH7AeOB5Zn5zp4CEfFlYJVJhSRJkjR8tKwrVGauAz4F3AQ8RDH70wMRcUFEnFIWOw/4WEQsAK4GPpqZvbtLSZIkSRpmolOf27u6unLu3LlVhyFJkiS1tYi4JzO7NlfOlbclSZIkNczEQpIkSVLDTCwkSZIkNczEQpIkSVLDTCwkSZIkNczEQpIkSVLDTCwkSZIkNaxj17GIiOXAExWHsQPwXMUxdCrrtnms2+axbpvHum0O67V5rNvmsW6H3q6ZOXVzhTo2sRgOImLuQBYT0eBZt81j3TaPdds81m1zWK/NY902j3VbHbtCSZIkSWqYiYUkSZKkhplYNNflVQfQwazb5rFum8e6bR7rtjms1+axbpvHuq2IYywkSZIkNcwWC0mSJEkNM7EYAhFxYkQ8EhG1iPhiH/u3ioh/KPffFRG7tT7K9hMRMyPitoh4KCIeiIj/00eZoyPipYiYX/78QRWxtqOIWBIR95X1NreP/RERl5b37cKIOKSKONtNROxTdz/Oj4iXI+Izvcp43w5QRHw7Ip6NiPvrtk2JiFsi4rHydXI/x55dlnksIs5uXdTDXz/1+qcR8XD57/3aiNiun2M3+d0x0vVTt1+OiKfq/s2f3M+xm3yeGOn6qdt/qKvXJRExv59jvW9bwK5QDYqI0cCjwHuAbuBu4MzMfLCuzG8AB2XmJyLiDOC0zPxgJQG3kYiYBkzLzHkRMRG4B/jlXnV7NPC5zHxfRWG2rYhYAnRlZp9zfZf/4/s0cDJwGPAXmXlY6yJsf+X3w1PAYZn5RN32o/G+HZCIeBewCvhOZh5QbvsqsCIzLy4fviZn5hd6HTcFmAt0AUnx/fELmflCS3+BYaqfej0e+GFmrouI/wvQu17LckvYxHfHSNdP3X4ZWJWZl2ziuM0+T4x0fdVtr/1fA17KzAv62LcE79ums8WicYcCtcxcXv8amgAABthJREFUnJlrge8Bp/YqcypwZfn++8BxEREtjLEtZebTmTmvfL8SeAiYXm1UI8qpFF/emZl3AtuVyZ4G7jhgUX1SocHJzB8DK3ptrv9OvRL45T4OPQG4JTNXlMnELcCJTQu0zfRVr5l5c2auKz/eCcxoeWAdoJ97diAG8jwxom2qbsvnql8Frm5pUHoTE4vGTQeW1n3uZuOH35+XKb+0XwK2b0l0HaLsPvY24K4+dh8eEQsi4oaIeGtLA2tvCdwcEfdExLl97B/Iva1NO4P+/yfnfbvldsrMp6H4AwSwYx9lvH8b8+vADf3s29x3h/r2qbKb2bf76b7nPduYdwLPZOZj/ez3vm0BE4vG9dXy0Lt/2UDKqB8RMQH4J+Azmflyr93zKJaZPxj4f8C/tDq+NnZEZh4CnAT8ZtnEXM/7tgERMQ44BfjHPnZ73zaf9+8WiojfA9YBf99Pkc19d2hjfwXsAcwBnga+1kcZ79nGnMmmWyu8b1vAxKJx3cDMus8zgGX9lYmIMcAktqyZdMSJiLEUScXfZ+Y/996fmS9n5qry/fXA2IjYocVhtqXMXFa+PgtcS9EMX28g97b6dxIwLzOf6b3D+7Zhz/R0yytfn+2jjPfvFigHub8POCv7GYQ5gO8O9ZKZz2Tm+szcAPx/+q4z79ktVD5b/QrwD/2V8b5tDROLxt0N7BURs8u/UJ4BXNerzHVAz4wkp1MMjvOvEJtR9pf8G+ChzPyzfsrs3DNeJSIOpbinn29dlO0pIrYpB8QTEdsAxwP39yp2HfBrUfhFigFxT7c41HbW71/PvG8bVv+dejbwgz7K3AQcHxGTy24nx5fb1I+IOBH4AnBKZr7aT5mBfHeol17j006j7zobyPOE+vZu4OHM7O5rp/dt64ypOoB2V86e8SmK/2GNBr6dmQ9ExAXA3My8juLh+LsRUaNoqTijuojbyhHAR4D76qaP+11gFkBm/jVFovbJiFgHrAbOMGkbkJ2Aa8tn2zHAVZl5Y0R8An5et9dTzAhVA14F/ldFsbadiHgLxcwuH6/bVl+33rcDFBFXA0cDO0REN/CHwMXANRFxDvAk8IGybBfwicz835m5IiL+mOJhDeCCzLSluNRPvX4J2Aq4pfxuuLOczXAX4FuZeTL9fHdU8CsMW/3U7dERMYeia9MSyu+G+rrt73migl9h2OqrbjPzb+hjPJv3bTWcblaSJElSw+wKJUmSJKlhJhaSJEmSGmZiIUmSJKlhJhaSJEmSGmZiIUmSJKlhJhaSpGEtInaLiCynk5UkDVMmFpIkSZIaZmIhSZIkqWEmFpKkTYrC5yNiUUSsjoj7IuLD5b6ebkofiojbI2JNRDwcEcf3Ose7IuKucv8zEfH1iBjX6xrnRcRjEfFaRHRHxEW9Qtk1Im6JiFcj4sGIeE/d8WMj4tKIWFYevzQiLm5qxUiS3sTEQpK0OV8BzgF+E9gfuAj4ZkS8t67MV4FLgTnALcAPImI6QPl6A3Av8LbyXGeW5+nxJ8Dvl9veCnwAWNorjgvLaxwM3A18LyImlPt+CzgNOAPYC/gg8EiDv7ckaRAiM6uOQZI0TEXENsBzwPGZ+ZO67X8O7A38BvA4cH5mXljuGwU8DFyTmedHxIUUD/p7Z+aGssxHgW8Ckyn+yPUc8JnM/Os+YtitvMYnMvOb5bbpQDfwzsy8PSIupUhI3p3+j02SKjGm6gAkScPa/sB44MaIqH9gHwssqft8R8+bzNwQEXeVxwLsB9zRk1SUbgfGAXuW598KuHUzsSyse7+sfN2xfL2CoqXk0Yi4GbgeuKHXNSVJTWRiIUnalJ4us78EPNlr3+tADOAcAfTXipADPEfP9YqDMjMifh5fZs4rWzZOBI4FrgQWRMR7TC4kqTUcYyFJ2pQHgdeAXTOz1uvnibpyv9jzJoon/kOBh+rOcXjZRarHkcBaYFHdNY5rJNDMXJmZ/5iZnwTeS5Fg7NnIOSVJA2eLhSSpX5m5MiIuAS4pE4YfAxMoEokNwM1l0U9GxKPAfRTjLnYF/qrcdxnwGeCyiPgLYHfgYuAvM/NVgHL7RRHxWnmN7YFfyMyec2xSRHwWeBqYT9Gy8SHgZYpxGJKkFjCxkCRtzu8DzwCfo0gWXqZ4gP9qXZkvAp8FDgGeAE7LzG6AzHwqIk4C/rQ87kXgKuB3647/EvBCea0Z5fW+M4gYVwK/QzEjVFLMQHVST+IiSWo+Z4WSJG2xuhmb3p6Zc6uNRpJUJcdYSJIkSWqYiYUkSZKkhtkVSpIkSVLDbLGQJEmS1DATC0mSJEkNM7GQJEmS1DATC0mSJEkNM7GQJEmS1DATC0mSJEkN+x9gByAIjyEOawAAAABJRU5ErkJggg==\n",
            "text/plain": "<Figure size 936x360 with 1 Axes>"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Download the saved model"
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "In the training script, a TensorFlow `saver` object is used to persist the model in a local folder (local to the compute target). The model was saved to the `./outputs` folder on the disk of the Batch AI cluster node where the job is run. Azure ML automatically uploaded anything written in the `./outputs` folder into run history file store. Subsequently, we can use the `Run` object to download the model files the `saver` object saved. They are under the the `outputs/model` folder in the run history file store, and are downloaded into a local folder named `model`. Note the TensorFlow model consists of four files in binary format and they are not human-readable."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# create a model folder in the current directory\nos.makedirs('./model', exist_ok=True)\n\nfor f in run.get_file_names():\n    if f.startswith('outputs/model'):\n        output_file_path = os.path.join('./model', f.split('/')[-1])\n        print('Downloading from {} to {} ...'.format(f, output_file_path))\n        run.download_file(name=f, output_file_path=output_file_path)",
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Downloading from outputs/model/checkpoint to ./model/checkpoint ...\nDownloading from outputs/model/mnist-tf.model.data-00000-of-00001 to ./model/mnist-tf.model.data-00000-of-00001 ...\nDownloading from outputs/model/mnist-tf.model.index to ./model/mnist-tf.model.index ...\nDownloading from outputs/model/mnist-tf.model.meta to ./model/mnist-tf.model.meta ...\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Predict on the test set\nNow load the saved TensorFlow graph, and list all operations under the `network` scope. This way we can discover the input tensor `network/X:0` and the output tensor `network/output/MatMul:0`, and use them in the scoring script in the next step.\n\nNote: if your local TensorFlow version is different than the version running in the cluster where the model is trained, you might see a \"compiletime version mismatch\" warning. You can ignore it."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import tensorflow as tf\n\ntf.reset_default_graph()\n\nsaver = tf.train.import_meta_graph(\"./model/mnist-tf.model.meta\")\ngraph = tf.get_default_graph()\n\nfor op in graph.get_operations():\n    if op.name.startswith('network'):\n        print(op.name)",
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "text": "network/X\nnetwork/y\nnetwork/h1/MatMul\nnetwork/h1/BiasAdd\nnetwork/h1/Relu\nnetwork/h2/MatMul\nnetwork/h2/BiasAdd\nnetwork/h2/Relu\nnetwork/output/MatMul\nnetwork/output/BiasAdd\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Feed test dataset to the persisted model to get predictions."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# input tensor. this is an array of 784 elements, each representing the intensity of a pixel in the digit image.\nX = tf.get_default_graph().get_tensor_by_name(\"network/X:0\")\n# output tensor. this is an array of 10 elements, each representing the probability of predicted value of the digit.\noutput = tf.get_default_graph().get_tensor_by_name(\"network/output/MatMul:0\")\n\nwith tf.Session() as sess:\n    saver.restore(sess, './model/mnist-tf.model')\n    k = output.eval(feed_dict={X : X_test})\n# get the prediction, which is the index of the element that has the largest probability value.\ny_hat = np.argmax(k, axis=1)\n\n# print the first 30 labels and predictions\nprint('labels:  \\t', y_test[:30])\nprint('predictions:\\t', y_hat[:30])",
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "stream",
          "text": "INFO:tensorflow:Restoring parameters from ./model/mnist-tf.model\nlabels:  \t [7 2 1 0 4 1 4 9 5 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1]\npredictions:\t [7 2 1 0 4 1 4 9 6 9 0 6 9 0 1 5 9 7 3 4 9 6 6 5 4 0 7 4 0 1]\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Calculate the overall accuracy by comparing the predicted value against the test set."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "print(\"Accuracy on the test set:\", np.average(y_hat == y_test))",
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Accuracy on the test set: 0.9726\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Intelligent hyperparameter tuning\nWe have trained the model with one set of hyperparameters, now let's how we can do hyperparameter tuning by launching multiple runs on the cluster. First let's define the parameter space using random sampling."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.train.hyperdrive import *\n\nps = RandomParameterSampling(\n    {\n        '--batch-size': choice(25, 50, 100),\n        '--first-layer-neurons': choice(10, 50, 200, 300, 500),\n        '--second-layer-neurons': choice(10, 50, 200, 500),\n        '--learning-rate': loguniform(-6, -1)\n    }\n)",
      "execution_count": 42,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Next, we will create a new estimator without the above parameters since they will be passed in later. Note we still need to keep the `data-folder` parameter since that's not a hyperparamter we will sweep."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "est = TensorFlow(source_directory=script_folder,\n                 script_params={'--data-folder': ws.get_default_datastore().as_mount()},\n                 compute_target=compute_target,\n                 entry_script='tf_mnist.py', \n                 use_gpu=True)",
      "execution_count": 43,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now we will define an early termnination policy. The `BanditPolicy` basically states to check the job every 2 iterations. If the primary metric (defined later) falls outside of the top 10% range, Azure ML terminate the job. This saves us from continuing to explore hyperparameters that don't show promise of helping reach our target metric."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "policy = BanditPolicy(evaluation_interval=2, slack_factor=0.1)",
      "execution_count": 44,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now we are ready to configure a run configuration object, and specify the primary metric `validation_acc` that's recorded in your training runs. If you go back to visit the training script, you will notice that this value is being logged after every epoch (a full batch set). We also want to tell the service that we are looking to maximizing this value. We also set the number of samples to 20, and maximal concurrent job to 4, which is the same as the number of nodes in our computer cluster."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "htc = HyperDriveRunConfig(estimator=est, \n                          hyperparameter_sampling=ps, \n                          policy=policy, \n                          primary_metric_name='validation_acc', \n                          primary_metric_goal=PrimaryMetricGoal.MAXIMIZE, \n                          max_total_runs=8,\n                          max_concurrent_runs=4)",
      "execution_count": 45,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Finally, let's launch the hyperparameter tuning job."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "htr = exp.submit(config=htc)",
      "execution_count": 46,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "We can use a run history widget to show the progress. Be patient as this might take a while to complete."
    },
    {
      "metadata": {
        "trusted": true,
        "scrolled": true
      },
      "cell_type": "code",
      "source": "RunDetails(htr).show()",
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ec61d34e586244598b891d660b64f048",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": "_HyperDriveWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': True, 'log_level': 'INFO',…"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "htr.wait_for_completion(show_output=True)",
      "execution_count": 48,
      "outputs": [
        {
          "output_type": "stream",
          "text": "RunId: tf-mnist_1545831390279\n\nExecution Summary\n=================\nRunId: tf-mnist_1545831390279\n\n",
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "execution_count": 48,
          "data": {
            "text/plain": "{'runId': 'tf-mnist_1545831390279',\n 'target': 'gpucluster',\n 'status': 'Completed',\n 'endTimeUtc': '2018-12-26T13:45:15.000Z',\n 'properties': {'primary_metric_config': '{\"name\": \"validation_acc\", \"goal\": \"maximize\"}',\n  'runTemplate': 'HyperDrive',\n  'azureml.runsource': 'hyperdrive'},\n 'logFiles': {'azureml-logs/hyperdrive.txt': 'https://amlsworkspace9663571855.blob.core.windows.net/azureml/ExperimentRun/tf-mnist_1545831390279/azureml-logs/hyperdrive.txt?sv=2018-03-28&sr=b&sig=Y%2FI5g6xb3N4ota5jSWrtkuz5R%2FEpu3UoRiZ266Kqu%2F0%3D&st=2018-12-26T13%3A35%3A45Z&se=2018-12-26T21%3A45%3A45Z&sp=r'}}"
          },
          "metadata": {}
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Find and register best model\nWhen all the jobs finish, we can find out the one that has the highest accuracy."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "best_run = htr.get_best_run_by_primary_metric()",
      "execution_count": 49,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Now let's list the model files uploaded during the run."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "print(best_run.get_file_names())",
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "stream",
          "text": "['azureml-logs/60_control_log.txt', 'azureml-logs/80_driver_log.txt', 'outputs/model/checkpoint', 'outputs/model/mnist-tf.model.data-00000-of-00001', 'outputs/model/mnist-tf.model.meta', 'outputs/model/mnist-tf.model.index', 'driver_log', 'azureml-logs/azureml.log', 'azureml-logs/55_batchai_execution.txt']\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "We can then register the folder (and all files in it) as a model named `tf-dnn-mnist` under the workspace for deployment."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "model = best_run.register_model(model_name='tf-dnn-mnist', model_path='outputs/model')",
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Deprecated, use RunHistoryFacade.assets instead.\n",
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Deploy the model in ACI\nNow we are ready to deploy the model as a web service running in Azure Container Instance [ACI](https://azure.microsoft.com/en-us/services/container-instances/). Azure Machine Learning accomplishes this by constructing a Docker image with the scoring logic and model baked in.\n### Create score.py\nFirst, we will create a scoring script that will be invoked by the web service call. \n\n* Note that the scoring script must have two required functions, `init()` and `run(input_data)`. \n  * In `init()` function, you typically load the model into a global object. This function is executed only once when the Docker container is started. \n  * In `run(input_data)` function, the model is used to predict a value based on the input data. The input and output to `run` typically use JSON as serialization and de-serialization format but you are not limited to that."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "%%writefile score.py\nimport json\nimport numpy as np\nimport os\nimport tensorflow as tf\n\nfrom azureml.core.model import Model\n\ndef init():\n    global X, output, sess\n    tf.reset_default_graph()\n    model_root = Model.get_model_path('tf-dnn-mnist')\n    saver = tf.train.import_meta_graph(os.path.join(model_root, 'mnist-tf.model.meta'))\n    X = tf.get_default_graph().get_tensor_by_name(\"network/X:0\")\n    output = tf.get_default_graph().get_tensor_by_name(\"network/output/MatMul:0\")\n    \n    sess = tf.Session()\n    saver.restore(sess, os.path.join(model_root, 'mnist-tf.model'))\n\ndef run(raw_data):\n    data = np.array(json.loads(raw_data)['data'])\n    # make prediction\n    out = output.eval(session=sess, feed_dict={X: data})\n    y_hat = np.argmax(out, axis=1)\n    return y_hat.tolist()",
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Writing score.py\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Create myenv.yml\nWe also need to create an environment file so that Azure Machine Learning can install the necessary packages in the Docker image which are required by your scoring script. In this case, we need to specify packages `numpy`, `tensorflow`."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core.runconfig import CondaDependencies\n\ncd = CondaDependencies.create()\ncd.add_conda_package('numpy')\ncd.add_tensorflow_conda_package()\ncd.save_to_file(base_directory='./', conda_file_path='myenv.yml')\n\nprint(cd.serialize_to_string())",
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "stream",
          "text": "# Conda environment specification. The dependencies defined in this file will\r\n# be automatically provisioned for runs with userManagedDependencies=False.\r\n\n# Details about the Conda environment file format:\r\n# https://conda.io/docs/user-guide/tasks/manage-environments.html#create-env-file-manually\r\n\nname: project_environment\ndependencies:\n  # The python interpreter version.\r\n  # Currently Azure ML only supports 3.5.2 and later.\r\n- python=3.6.2\n\n- pip:\n  - azureml-defaults==1.0.2\n- numpy\n- tensorflow=1.10.0\n\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Deploy to ACI\nWe are almost ready to deploy. Create a deployment configuration and specify the number of CPUs and gigbyte of RAM needed for your ACI container. "
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core.webservice import AciWebservice\n\naciconfig = AciWebservice.deploy_configuration(cpu_cores=1, \n                                               memory_gb=1, \n                                               tags={'name':'mnist', 'framework': 'TensorFlow DNN'},\n                                               description='Tensorflow DNN on MNIST')",
      "execution_count": 54,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "#### Deployment Process\nNow we can deploy. **This cell will run for about 7-8 minutes**. Behind the scene, it will do the following:\n1. **Register model**  \nTake the local `model` folder (which contains our previously downloaded trained model files) and register it (and the files inside that folder) as a model named `model` under the workspace. Azure ML will register the model directory or model file(s) we specify to the `model_paths` parameter of the `Webservice.deploy` call.\n2. **Build Docker image**  \nBuild a Docker image using the scoring file (`score.py`), the environment file (`myenv.yml`), and the `model` folder containing the TensorFlow model files. \n3. **Register image**    \nRegister that image under the workspace. \n4. **Ship to ACI**    \nAnd finally ship the image to the ACI infrastructure, start up a container in ACI using that image, and expose an HTTP endpoint to accept REST client calls."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from azureml.core.image import ContainerImage\n\nimgconfig = ContainerImage.image_configuration(execution_script=\"score.py\", \n                                               runtime=\"python\", \n                                               conda_file=\"myenv.yml\")",
      "execution_count": 55,
      "outputs": []
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "%%time\nfrom azureml.core.webservice import Webservice\n\nservice = Webservice.deploy_from_model(workspace=ws,\n                                       name='tf-mnist-svc',\n                                       deployment_config=aciconfig,\n                                       models=[model],\n                                       image_config=imgconfig)\n\nservice.wait_for_deployment(show_output=True)",
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Creating image\nImage creation operation finished for image tf-mnist-svc:1, operation \"Succeeded\"\nCreating service\nRunning....\nSucceededACI service creation operation finished, operation \"Succeeded\"\nCPU times: user 27.4 s, sys: 19 s, total: 46.4 s\nWall time: 11min 8s\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "**Tip: If something goes wrong with the deployment, the first thing to look at is the logs from the service by running the following command:**"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "print(service.get_logs())",
      "execution_count": 57,
      "outputs": [
        {
          "output_type": "stream",
          "text": "2018-12-26T14:05:13,175760628+00:00 - rsyslog/run \n2018-12-26T14:05:13,175692428+00:00 - gunicorn/run \n2018-12-26T14:05:13,176239829+00:00 - nginx/run \n2018-12-26T14:05:13,179474035+00:00 - iot-server/run \nok: run: rsyslog: (pid 11) 0s\nok: run: gunicorn: (pid 10) 0s\nok: run: nginx: (pid 9) 0s\nok: run: rsyslog: (pid 11) 0s\nok: run: rsyslog: (pid 11) 0s\nEdgeHubConnectionString and IOTEDGE_IOTHUBHOSTNAME are not set. Exiting...\n2018-12-26T14:05:13,320212190+00:00 - iot-server/finish 1 0\n2018-12-26T14:05:13,321331392+00:00 - Exit code 1 is normal. Not restarting iot-server.\n{\"timestamp\": \"2018-12-26T14:05:13.544856Z\", \"message\": \"Starting gunicorn 19.6.0\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/gunicorn/glogging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"gunicorn.error\", \"msg\": \"Starting gunicorn %s\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:13.545661Z\", \"message\": \"Listening at: http://127.0.0.1:9090 (10)\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/gunicorn/glogging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"gunicorn.error\", \"msg\": \"Listening at: %s (%s)\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:13.545761Z\", \"message\": \"Using worker: sync\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/gunicorn/glogging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"gunicorn.error\", \"msg\": \"Using worker: %s\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:13.546360Z\", \"message\": \"worker timeout is set to 300\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/gunicorn/glogging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"gunicorn.error\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:13.547350Z\", \"message\": \"Booting worker with pid: 38\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/gunicorn/glogging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"gunicorn.error\", \"msg\": \"Booting worker with pid: %s\", \"stack_info\": null}\nInitializing logger\n{\"timestamp\": \"2018-12-26T14:05:18.502887Z\", \"message\": \"{\\\"requestId\\\": \\\"00000000-0000-0000-0000-000000000000\\\", \\\"message\\\": \\\"Starting up app insights client\\\", \\\"apiName\\\": \\\"\\\"}\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/var/azureml-app/aml_logger.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"root\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:18.503167Z\", \"message\": \"{\\\"requestId\\\": \\\"00000000-0000-0000-0000-000000000000\\\", \\\"message\\\": \\\"Starting up request id generator\\\", \\\"apiName\\\": \\\"\\\"}\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/var/azureml-app/aml_logger.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"root\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:18.503295Z\", \"message\": \"{\\\"requestId\\\": \\\"00000000-0000-0000-0000-000000000000\\\", \\\"message\\\": \\\"Starting up app insight hooks\\\", \\\"apiName\\\": \\\"\\\"}\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/var/azureml-app/aml_logger.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"root\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:18.503390Z\", \"message\": \"{\\\"requestId\\\": \\\"00000000-0000-0000-0000-000000000000\\\", \\\"message\\\": \\\"Invoking user's init function\\\", \\\"apiName\\\": \\\"\\\"}\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/var/azureml-app/aml_logger.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"root\", \"stack_info\": null}\n2018-12-26 14:05:18,504 | azureml.core.run | DEBUG | Could not load run context Failed to load a submitted run, if outside of an execution context, use project.start_run to initialize an azureml.core.Run., switching offline: False\n2018-12-26 14:05:18,504 | azureml.core.run | DEBUG | Could not load the run context and allow_offline set to False\n2018-12-26 14:05:18,504 | azureml.core.model | DEBUG | RunEnvironmentException: Failed to load a submitted run, if outside of an execution context, use project.start_run to initialize an azureml.core.Run.\n2018-12-26 14:05:18,504 | azureml.core.model | DEBUG | version is None. Latest version is 1\n2018-12-26 14:05:18,504 | azureml.core.model | DEBUG | Found model path at azureml-models/tf-dnn-mnist/1/model\n2018-12-26 14:05:18.536548: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA\n2018-12-26 14:05:18.537407: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.\n{\"timestamp\": \"2018-12-26T14:05:18.538622Z\", \"message\": \"Restoring parameters from azureml-models/tf-dnn-mnist/1/model/mnist-tf.model\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/tensorflow/python/platform/tf_logging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"tensorflow\", \"msg\": \"Restoring parameters from %s\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:18.551952Z\", \"message\": \"{\\\"requestId\\\": \\\"00000000-0000-0000-0000-000000000000\\\", \\\"message\\\": \\\"Users's init has completed successfully\\\", \\\"apiName\\\": \\\"\\\"}\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/var/azureml-app/aml_logger.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"root\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:18.552624Z\", \"message\": \"{\\\"requestId\\\": \\\"00000000-0000-0000-0000-000000000000\\\", \\\"message\\\": \\\"Scoring timeout setting is not found. Use default timeout: 3600000 ms\\\", \\\"apiName\\\": \\\"\\\"}\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/var/azureml-app/aml_logger.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"root\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:05:24.312025Z\", \"message\": \"127.0.0.1 - - [26/Dec/2018:14:05:24 +0000] \\\"GET / HTTP/1.0\\\" 200 7 \\\"-\\\" \\\"Go-http-client/1.1\\\"\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/gunicorn/glogging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"gunicorn.access\", \"stack_info\": null}\n{\"timestamp\": \"2018-12-26T14:06:04.002941Z\", \"message\": \"127.0.0.1 - - [26/Dec/2018:14:06:03 +0000] \\\"GET / HTTP/1.0\\\" 200 7 \\\"-\\\" \\\"Go-http-client/1.1\\\"\", \"host\": \"wk-caas-875c7c79c5ec405ea1a5284fbc20fbb3-480605903cfa8aa0460fd4\", \"path\": \"/opt/miniconda/lib/python3.6/site-packages/gunicorn/glogging.py\", \"tags\": \"%(module)s, %(asctime)s, %(levelname)s, %(message)s\", \"level\": \"INFO\", \"logger\": \"gunicorn.access\", \"stack_info\": null}\n\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "This is the scoring web service endpoint:"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "print(service.scoring_uri)",
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "text": "http://52.142.234.83:80/score\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "### Test the deployed model\nLet's test the deployed model. Pick 30 random samples from the test set, and send it to the web service hosted in ACI. Note here we are using the `run` API in the SDK to invoke the service. You can also make raw HTTP calls using any HTTP tool such as curl.\n\nAfter the invocation, we print the returned predictions and plot them along with the input images. Use red font color and inversed image (white on black) to highlight the misclassified samples. Note since the model accuracy is pretty high, you might have to run the below cell a few times before you can see a misclassified sample."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import json\n\n# find 30 random samples from test set\nn = 30\nsample_indices = np.random.permutation(X_test.shape[0])[0:n]\n\ntest_samples = json.dumps({\"data\": X_test[sample_indices].tolist()})\ntest_samples = bytes(test_samples, encoding='utf8')\n\n# predict using the deployed model\nresult = service.run(input_data=test_samples)\n\n# compare actual value vs. the predicted values:\ni = 0\nplt.figure(figsize = (20, 1))\n\nfor s in sample_indices:\n    plt.subplot(1, n, i + 1)\n    plt.axhline('')\n    plt.axvline('')\n    \n    # use different color for misclassified sample\n    font_color = 'red' if y_test[s] != result[i] else 'black'\n    clr_map = plt.cm.gray if y_test[s] != result[i] else plt.cm.Greys\n    \n    plt.text(x=10, y=-10, s=y_hat[s], fontsize=18, color=font_color)\n    plt.imshow(X_test[s].reshape(28, 28), cmap=clr_map)\n    \n    i = i + 1\nplt.show()",
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABHEAAABJCAYAAAC6jrqSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzsnXdYFMcbx79zgAIWBCsakphYMGosGHvvGMWuP8USS4ioMfaGBUsw2Bt2TYhdorGioih2VKKi0XhBQcECKlY6x83vj2Und8eBILe3SzKf57nnYPe4/TI7O+Wdd96XUErB4XA4HA6Hw+FwOBwOh8NRNiq5BXA4HA6Hw+FwOBwOh8PhcN4PN+JwOBwOh8PhcDgcDofD4RQAuBGHw+FwOBwOh8PhcDgcDqcAwI04HA6Hw+FwOBwOh8PhcDgFAG7E4XA4HA6Hw+FwOBwOh8MpAHAjDofD4XA4HA6Hw+FwOBxOAYAbcTgcDofD4XA4HA6Hw+FwCgDmMeIQQrN5JZjl+kYlkbKEkHWEkBhCSBohJJoQsoIQUkJGTd6EEJrDK10mXUosq+zKSLY6pXBdDoSQxYSQe4SQFELIc0LIaUJIM5n0VCWEbCeE/EUIeUMISSKE3CWELCWEOMqkqQohZC4hJDSzfN4RQm4QQrwIIUW4Jj1dSm2rihJCphNCbmWW1QtCyEVCyDeEECKHJkMIIbaEkKjMclotow4VIWRc5nOXktm+L5GzXmXqUlQbqsS2KlOXosopU5Mi2ytDlPAMEkKmEUICCCGRmToeyKHDQJMi67ohSrh/mToU2d8oabyn5Dql0GdQqf2youaBiq1XhJQFIetASAwISQMh0SBkBSQqJ0spvjQbzgHYYHBMroF+GQCXAZQHsB7AnwBqAPAE0JwQ0oRSmiSDtH0A7hk5/iWASQAOmVeOossKUFCdMkBRugghnwAIAVAUwGYAfwOwg1CvKsgk6yMAjgB+B/AIgAZATQAeAP5HCKlNKX1mZk1DAYwCcBDAdgj3rBWA+QD6EEIaUkqTuSYAymyrVACOAmgMwB/AKgC2APoB+BlANQBTzK3LCHMBlJJbBIBlAMZAeAaXQCifMQDqEELaUkq1MmpTUhuqxLZKREnlBCi3vTJECc+gD4CXAK4BkG1BzAAl13VdZL9/Su1vFDjeU3KdUuIzqLh+WaHzQOXVq/eUEwhpAlOXE6VU+hdAKfCLWa6VixeA5QAogH4Gx/tlHp8ht0YDXeszdX3Ny4pdn0JBdUrJuiAM9GMAOMqtJRdae2eW4WQZrl0PgJ2R4/MzNY3mmt6rV862qlHmtZcZHC8EIBLAawWUT10Ig43xmVpXy6SjOgAtgL0Gx7/P1NVfxjJSXBuajU7Z2iqlllNBaK8U9Ax+pvPznwAeyF02OWiVta4r9P4psr8pKOM9JdQppT2DSu2XodB5YDZa5atXwPJMe0c/g+P9Mo+bvJzMGxOHkEIgpKhZr2mcVgCSAewyOL4bQAqAIWZXlA2EEFsA/wPwGMAxGSQouqwIIYWIMuqUHkrRRQhpDqApgIWU0qeEEKvMOqVUHma+25v7wpTSMErpGyOndme+1zCnHkCZmrJDAW1V8cz3J7oHKaVpAF4ASDS7Ih0IIRYANkIom31yaoEw+CIQBme6bASQBGCA2RUZoJQ2NAdka6t0UVI5Kb29UtIzSCmNlPP6eUQpdV0x9w8K7G8K2HhP9jqlwGdQqf2youeBBshZr8xeTuY04vSCUAnfgZBnIGQVCLEz4/V1KQwghWaa7USo4KaWDOAzQojcrrYifSB0Fj9TSjNkuL6Sy4rVKULIM0LIKiJfndJFSbo6Zb5HE0IOQbhniYSQvwkhSpioWRNCShFCPiKEtIfgyQEAgXLqMuCjzPc4WVXoo0RNcrdVVwC8BjCZENKbEPJx5r7pBQBcAHjLoEmXcQCcAYyWWQcAfAVhxe+K7kFKaQqAG5nn5URJbSgAxbZViiunbFBKe6WkZ1CxKLSuA8q6f0rsbxQ73lNwnVISSu2XFTsPVFi9KgwgBQblBJ1ygonLyVwxca4ACIAQQ6E4hIZmNIAWIKQxKDV3IL7bAKpm7pm7IR4khNTGP9a7jyFY0+VmGATXsC0yXV+pZZVtnSKENKbmr1NK1VU1830jgAgAgyE0NOMBbCWEWFFKfzazJl2GQ9hLLvIAwABK6Tl55OiTufI3C4L79g6Z5QBQpqZMZG2rKKWvCCFuADYB2KNz6h2AnpTS/XLoAgBCSEUAcwDMpZQ+IIR8KpeWTMoDeEEpTTVy7jGAxoSQQpmryuZGaW2oiNLaKqWWkx5Kaa8U+AwqGaXVdcXdP4X2N0oe7ymuTikQpfbLSp0HAsqqV7cBVAUhtaFTTpCwnMxjxKG0gcGRX0HITQA/Avgh892cLAfQDcAeQshYCHshq2ceTwdgBSFAmawQQqpCcI0MppRGySRDkWVFjdQpIm+dAqBIXcUy398BaCU2/oSQ3yHs2/YhhPhT+YKY7gdwF0IQvjoA3ACUlkmLMZYDaAhgOqVULbeYTBSnSSFtFQAkQGijDgK4CMABQqDVHYSQrpTSEzLpWgsgCsBSma5viC0AYwNFQHD7FT9jdiOOAttQEUW1VQouJ0OU0l4p7RlUMoqq65ko8f4prb9R8nhPiXVKaSi1X1bkPDATJdUrVk4wVzmZPfDPP4F+rCiQSoGLclwfQvCjpxBWjimEVaJ1EPbZUgBfylY2/2hcmKnlfzLrUHxZZeq0gtAAylKnlKgLQpYgCmC+kXP+meeqyV1GOpq+zCyraQrQMi+zfNbLrUXJmjJ1yd5WQchMkAxghMFxWwgD2AcALGTQNQCCi3RTnWOfQt6gnLcAxGVzbk+mtkJy1ikDTYpr25XUVim1nJTSXinxGTTQJ3tQ1ffok7WuK/H+KbG/KUjjPbnrlBE9sj+DSu6XUXDmgfLWK6A3BZ5mBjKmFNBQYB0F9mX+btJyMm9gY10oTYcQEEyWfXSU0gAI+7TrAGgOoDyldETmMQ2Mp881G4QQSwCDIKS/+11OLUovKxEqc53KDpl1Pcp8jzVy7mnmu6zBCnWhlN4EcB3ASDl1EEK8AcyAkCp0hJxaRJSoCVBUWzUOgDWELSYMKqR0PALgEwgDf7NBCCkMYeU4EEAsIaQSIaRSphYAsMs8Zu70pk8AlMrUZ0gFCC7dcmylMooS23altFW6KKmclNJeKfgZLDDIWdcVfP8U19+gAI33lNh+KgDF9ssFaB4ob70yUk6QsJzkM+IQYg3hn5It0B2lNINSeoNSeo5S+owQUg5CwZ+h5s95b0gXAGUBbKXG90eaFYWXFQAhwBVkrlPGkFmXGCDtIyPnxGPPzKQlt9hAcEuWBULIbACzAfwKYDjNNO/LiRI16aCUtqpC5ruFkXOWBu/mwgaCa+/XEGIUiK+QzPMDMn8fbmZdVyH0//V1D2a2VbUBhJlZT44otW2HzG2VIUopJ4W1V0p9BgsactV1pd4/JfY3BW28p6j2UwEoul8uCPPATOStV5RmgNIboPQcKH0GnXKCictJeiMOISWzOTMPQgN3SHINuYAQogKwEkKDrIS95MMy3zfLqsIIcpcVUWidUqiu/RD2Rw8gOmloCSGOEPZuRlBKzW5Bz2z8jR1vBSENbah5FbHrz4KQVWIrgCFUvlhBDCVqMkApbdWdzPdvdA9mrtB2BfAKwH0za0qE4IZs+BJXiY5l/n7QzLp2Q3CBHmtw/FsI2wG2m1kPAGW2oUpsq5RYTiIKbK+U+gwqDiXWdSj3/imxv1HceE+hdUqpKLJfNoYC5oEFo15JXE5E8gUSQpZBCGp3GkA0hOBDnSDkU78MoBUoTZZWhKEkUhSCxfp3CIHS7AD0g5AW0ItS6mNOPYYQQspDKKs/aNbghebWoriyIrmoU9TMdUrhujwgpN27DSFzUCEAngAcAXSmlAbJoOn3zOufAvAQgluyC4D/QUiZ25LqRnc3j6ZRAFZDuHczIezB1yWOmjlQoRI16aKwtuoTANcguItvB3ABwmrMtxDc2kdRStfIJlCHzMwqUQD8KKWypMslhKyCkM3odwhbFaoBGAOh3FrLMflWYhuq0LZKceWUqUvR7ZUucj+DhJCB+GdL0PcQ+uUlmb8/pJRulUGT4up6dijg/imyv1HaeE/JdUqhz6AS+2UlzgOVV6/eU06QopzMEOSnKwWOU+AxBVIokEiBGxSYTgFrya9vPPBRIQC7Mgs5BUIsh+MAOsihx4i+6RCssd8qQIviygrCKsdxCCn3UiCs1NzILDdZ6pSSdWVq6wHBMp0IYaUmCEATGfX0gbBvPCazrJIhRJhfBeBjmTT9gn+Cthl7hXBNWfQppq3K1PM5hACOjyBkA3gL4CyAHnJrM9D5KWQOqgphZWgCADWEQICPIcSeKCqjJsW1oQptqxRXTpm6FN1eGWiV9RmEsB1IUeWkxLqu1PuXqUGR/Q0UNN5Tcp1S6DOoxH5ZifNA5dUroBAFdlEgKtPe8TLT/iFZOUnvicPhcDgcDofD4XA4HA6Hw8k38gU25nA4HA6Hw+FwOBwOh8Ph5BpuxOFwOBwOh8PhcDgcDofDKQBwIw6Hw+FwOBwOh8PhcDgcTgGAG3E4HA6Hw+FwOBwOh8PhcAoA3IjD4XA4HA6Hw+FwOBwOh1MA4EYcDofD4XA4HA6Hw+FwOJwCADficDgcDofD4XA4HA6Hw+EUALgRh8PhcDgcDofD4XA4HA6nAMCNOBwOh8PhcDgcDofD4XA4BQDLvHzYwqY4VdkWl0pLjmhePgallBget7S1o1YlysLGysLsmlSJz/HixYssmgBBV/HS5c0tCUD2upSqSa77BwCvou++oJSWNjzOyyp3mgBeVgVBE8Dren41AbysCoImgNf1/GoCeFkVBE0Ar+v51QTwsioImgBe1/OrCeBllVtNgDJtHoa814hDCPEA4AEAVmU+g0Obb00gL+/E7ZxmVFPhcpVQyWMVvnA0f0Hf3/i93u+GutpN32J2TYC+roKgSa77BwB7RjR+mJ2uf1NZUUrx5MkT3L9/H/Xq1YOtre0HazKm699UVv9GTcB/p66bUpMxXbyslK0J4HX9QzQZ08XLStmaAF7XP0STMV3/1rLKyMjA06dPAQBxcXFISEgAADRq1BCFChWWRdOHwut63jUZ08XLyrgmQ11KsXnkCKU01y9LhwoUgGwvY5rsP3amfdZdpHLg4uJiVJOoSy6y06VUTXLdP0opBRCWnS65MHVZ/f3337Rdu3ZUpVLRO3fumFQT/ZeV1b9VE6X/jboupSbKy6pAaKL031/Xk5OTaY8ePSgA6ufnR1evXk1HjBhB1Wo1VavV+dZETVRWr1+/ptHR0dTLy4t6eXlRR0dHWq5cORoSEvJBunhdz0p+63pAQABdv349Xb9+Pe3WrRv7+eTJkx+sSYll9V9r11NTU+mpU6coIUTvVapUKerk5ESfPn1qdk355d/erkutifKyyrUmSpVp8zB88Zg4HA6Hw+FwOBwOh8PhcDgFgDzFxOFwlMzZs2cBAH5+fvjzzz8RFRWFxYsXY8SIEVCp/rv2ys2bNyM4OBgAUK1aNZnVKB+tVou4uDhs27YNy5YtQ1xcHDtevnx5XLlyBRUqVJDs+omJiXj+/DkAYPbs2QCAmjVrYuLEiZJdk8PhFBymT5+OAwcOQKVSYcyYMex4QEAAAOD+/fuws7OTSx4CAwOxfft2nD59Gs+ePQOlFABACAGlFP3798f169dRpkwZ2TT+13n06BFcXFxYXyNy4MABYYVXpYKlpTBFEMdWDRo0MLtOpfLs2TP89NNPOHr0KNRqNWxtbXH58mVUr15dbmlISUlBt27dcPz4cVhaWsLZ2RkAMHjwYHh4eKB4cdNuW4mMjERsbCx7vsV3AKhRo4bJr2cOfvrpJ0ybNg1+fn4YNGgQihYtanYNMTExOHToEI4fPw4ACAsLA6UUYWFhKF9enrg2crBmzRqMGjUq2/N+fn4YOXKkGRUpB27E4RR4EhISMGnSJPj7+wMAUlNT2bnvv/8eJUqUQN++fWFhYZ6gWenp6UzDu3fvsHXrVr3zx44dQ0hICADAy8sL8+bNk0xLWFgYli5dCgDo3bu3ZNfJK2/evMGJEyfg7++P8PBw1KhRA/Xr12cNdenSWWLXmY1Tp06hY8eO7HdChNhiKpUKcXFxaNq0KcLDwyUZmNy8eRMDBgxAqVKlAACXL19GcnIyAGDRokWIiIgokAMizj/Ex8ejSpUq6NatGwDByCoXL168wJAhQ3Dr1i0cP34cr1+/BgDs27cPs2fPznPsrH8DlFKcP38eN27cAACsXbsWf/31V7afnzNnDjp16oS6deuabbHAcOIt8urVKwDAhg0bMGnSJLNoMWTs2LFYuXKl0Qld2bJlQSlFbGwsdu3aBU9PT1hZWcmiMzw8HABQv3591K1bF5cuXZJFhy6PHz8GALi4uKBHjx5YuXIlM6SYmiVLlmRbj8R7lp6eDgBo1aoVAMF4OGPGDEn0FARSU1MxfPhwAMDhw4fx9u1bFC9eHBs2bMDGjRsxdOhQXL58WVaNd+/exfDhw3Hx4kWoVCosX748xwlwfvH09MSJEycQFRUFlUoFrVbL3gHA1dUV7u7u6Nevn2QaTInYLixcuJAZyTt16mRWI45Go8GSJUvg7e2tN58RadOmDW7cuIHChY3HNPo3IY6/c2LUqFEIDg6Gj48PqlatagZVysEsRhwnJyd8++236NatWxYrdb9+/bBnzx5zyODkgZSUFABAbGwsAgICEBoaigYNGsDJyUkRjbFWq8WoUaPw8OFD3Lhxg3lLGGPw4MHo2LEjHBwcTK4jMTERXl5eAP5pbO7du4fAwEA2gDVEPO7s7IyxY8eaXBMAJCUlARA60IyMDLRo0QK//vqrJNfKLcHBwTh27BgAYZIYFRXFzj169AjHjh2Dj48PAGDdunUYOnSo2TW+fv1ab8BTr149rFu3DoBQps2bN0dMTAy2bt1q8oHRzZs30bBhQ/bsAUDdunVRrVo1/PHHH7h79y4uX76Mdu3amfS6H8rr16+xefNmXLlyBQEBAbC1tYWHhwcAYWJbrFgxWXQlJCTg9u3bOH36tN5xDw+PLG1AcHAwunbtCgCYOnWq5BMUjUbDJmYDBgzQOxcbG4siRYqYrdyePXuGxo0bIyoqCpRSfPnll/jyyy8BANeuXcP58+dx4sSJ/4Qhh1KKtLQ0XLlyBUeOHMHChQv1zhcpUkRvMp2QkMAmKd7e3vD29kbLli0RFBQk2aQbAOvnwsLC2LGVK1eiXLly2LFjB37//XcAQJ06dSTTkBMREREIDAwEIQSEEJQtWxbDhg1jbVatWrVw7NgxuLu7Y9y4cejevTucnJzMrlOj0WDYsGEAhEWXBw8emF2DIS9fvsSIESMACPc5ODgYGo1Gkvp06NAhrFy5MtefF/skb29vADB5O+nq6oqwsDBcu3Ytz/Vh6NCh8Pf3x5YtWzB48GCT6tIlPT0dc+fOxY4dOwAAZcqUwZdffol169ahWrVqcHBwQO/evXH9+nUA8jyDMTExmDp1Ki5evAgAWLVqlVk8FDp06AA/Pz+9Y56engCAoKAg9O/fHwMGDEBERAQ+++wzyfV8KKmpqWwM+vbtW9l0bNiwAdOm/RPY9quvvgIgLMQ+f/4cixYtQuXKlREREfGvNuSsWbOG/dyjRw+0adNG77zuGHzfvn1o06aNIow4xYsXx7x581CtWjXMnTsXly5dQkZGhiTXkmy0ITbEo0ePxuDBg1GqVClm3f/jjz8wefJkAPqDEXOh0WiQkpICf39/xMXFYf78+XrnKaXo1q0bFi5ciMqVK5tdnyHiasjly5dx4MABLF68GLVq1cKlS5dgY2Nj8uv99ttvbBuHWq1mx/fv349SpUrh6tWrGDduHADIMgADgEGDBmHnzp0AYNRYUq9ePfTt2xeNGzdGiRIlJDHgAED37t1x8uRJALmzGANA37590aNHD3Tu3FmS+wcIHkiAMDAEgE6dOqFQoUKSXCs3vH79Gq6urqhYsSIAYNmyZQCEe+fo6IgiRYpg7ty5bDvAsmXLZDHidOzYEffu3WP3MiwsDNu3bwcguNcOHDgQW7duRWRkpEmvq2vA6d69O1asWAEAcHR0hIWFBW7fvo2GDRti4MCBOHDggGxu7QcOHAAAbNu2DSdPnmQrkbVq1cLNmzeZbjc3N7Rs2dLs+p4/f46hQ4fiyJEjWc598cUXcHNz0ztWs2ZNVKpUCYB5+qLx48dj9erVWLx4MVvdBgTPtPLly2Px4sUYP3685DoWL16MwMBAREVF4eOPP0b58uURGhrKtuwtWrQIoaGhcHV1xcmTJ2XzlsgNojeBnZ3dB7VxGo0G3t7ebPBevHhxlC5dGp9++ikAYOLEiWjfvr3e9qSQkBB23UmTJiE6OhohISG4ePEimjVrluu+IK+IxpCIiAgAwMWLF1lb0KNHD0mumRcmTZqE+/fvg1KKYcOGYcOGDVk+06hRIxaUUS6uXLmCa9eusd8PHz4smxaRNWvWMB2EEPj7+8Pa2lqSayUlJemV/7ffChlYSpQoofe558+f45dffmG/a7VabNq0yaRGnOvXr+Ps2bNITU1F3bp1s/UO0iU9PZ15mvn7+4MQgpEjR+oZ50zNpk2bsGDBArYl/cyZM8xjFhDGWE5OTnjz5o0k188JcdGuffv2bMz+559/4osvvpD82mvXrs3xeFRUFCpVqgSVSoV27drh5MmTbByoNB4/foy9e/fqHStatKikhnlDoqOj2fwKEAw4Z86cAQBYW1vj7du3CAwMxO3bt6FWq9nCi5SEhISw3QM3btxA7dq1MWzYMLYdVkpDkp+fX7bGmZEjR+pttTI08siBnZ0dpk+fzuZgbdu2xZAhQ9hOEQBwdnZG5cqV8fDhQ9y8eTNf15OkZn799ddsJUu34FNSUlC4cGEEBwezSmku3r17xyaIx48fx2+//QbAuAGAEIIDBw7g/PnzuHXrFsqVKyeJptTUVCQmJjL3vYsXL8LT0zNLxyla1c+fPw9A2NZx69YtJCcnm9wIoFar0bdvX1Ym3377LUaPHs3Ox8fHY86cOWjevDkA4K+//pJsoJEdd+/eZfdSxNraGg0bNgQA/Pjjj6hXr57kDe+1a9f06nGpUqXg7u6e7efFvchS72WNiorCtm3b2O/jxo3T6xTkoHjx4oiPj2eTQWN15uOPP2Y/N23a1GzadHny5EmWY+LzaWFhAXt7e0mu6+Pjg3r16iEiIgLbtm3L8lxXr14dGzZsgLu7O5YvX84MmOYgNTUVISEh2Lp1K1uFLFGiBNq3b48BAwagadOmKFq0KOrXr5/vDim/REdHMwPOvHnz9Oq94UBDo9EgICCAaRYnxlJx8OBBbNy4EUWKFEGLFi3Y8fj4eGZIMpxEmRpxa6WXlxc0Gg08PT0xa9YsrFq1ChUrVkSXLl0ACJ4nffv2xblz5xAVFYUqVapIqiu3iH3mvn378PPPPyMxMZFNWhwdHVGnTp0sg/Ds0PUuEMcrpUuXxu3bt/UmZ8bQNVBqtVrmndqyZUukp6dLsnX39OnTuHfvnt4xpcUoWbVqFWrWrIl27dqhUaNGRj8TFRXFvHTk2ho6depU9rOLiwvq1q1rku9NTEwEICye5GWB6/bt25g1axZr99u1a8fGM1JQrVo1tG3bFo8fP0ZQUBAb4xrWW41Gg5kzZzLjoakXLwDhORS3jLx69QoBAQHv3fqdnJyM1atX6x1LTU2Fr6+vJEacR48esYmiaJwwbCOsra2zrfNSkpSUxMadarUalStXxowZM1gcHLmpWLEiIiIi0K5dO0RGRiIgIIAt4iuN/fv3Zzk2YMAAfPTRR2a5/ps3bzB9+nS2cA8I3je64+XixYtjy5YtaNCgAWbPns28L01NSkoKpkyZAgDYuHEj0tLSAAhz5sOHD2P+/Plsbrh8+XJJNLzPi0ytVrPn0s/PT1YvHPEe7d27F61bt9Y7N2PGDGbEqVWrFoKDg+Hg4ICEhASULl3a6Ja53GLyWe6QIUMwbdo0oy5zTZo0wYABA/DDDz/g/v37AIBdu3axjk8q1Go1unTpwq4pUqlSJUyePDlLZxkaGgpfX1/cv38f169fh6urq8k1paSkYOvWrcx9VmTWrFm5/o4///yTGVNMwfXr19G4cWMAwqoDIGx3M5z8BAYGsoe3RYsWOHDggGSGLl3EAUSTJk2g0WgACA9OzZo1sWzZMrN2oK9fv8bAgQOZDgD4+eef0alTJ7NpMIZWq8WsWbOYrsKFC2PKlClmiweUHSqVKsdtIrdv38aKFSvYIFbcd25uypcvj1evXiEtLQ1ly5bF2LFjWV1PTU3FuXPnTL6KfPv2bdjb26NBgwbw9PTM1ihauXJllChRghmVzIG/vz98fHxw7949ODs74+DBgwCAZs2aZQmcWqRIEfZzvXr1zKZRRHQlL1myJKZNm5ajR8vdu3exdu1arFq1innpSTmwfPPmDQYPHoy0tDSEhITAxcWFnVu2bBlz3RaNKFJw4sQJ5mlDCEGjRo0wd+5cODg4ZInL5erqijNnzsDX1xdNmjTB5cuXZXGD12q1+PvvvwEInqg+Pj7sd0DY0iDGr2rUqJFekN/3IRpuxNgHLi4uWLp06XsNOACQlpbGVr/FgS4gDDqliItDKcWdO3fYQLpw4cJYsGCBya+TX5ycnN4b4+3EiROglMLZ2VnS4Mvi/dm1axcGDx7M+sCYmBhcvXqVfU6lUpnMc0psA3XbwveRkZGB6dOng1LKtuCIHo9S8eWXX+LgwYPQaDQ5xvq4dOkSvv/+e7x48UIyLevXr9f7PT+GIqli6fn6+oIQgu7du6NZs2ZGP5Oamqrn3WUujh07pldf5s+fr6j4hwDw2WefoWPHjli3bh28vLwUacS5dOlSljhiWq3WLO2s6DH/1VdfsTADnTp1wsOHD1nsPF3EsCT79+/H/fv38fnnn5tcU79+/XDo0CEAwnhh0aJF+O6775CUlAQfHx+sWrWKaRXjIJkTtVrNDJU9evSQNbBx8eLFmTFN18M6OTkZkZGROH36NFuoO3nyJBtzFi1aFCNHjmQQO8cHAAAgAElEQVQ7Ez4EkxpxhgwZAj8/PxQuXJitGJUvXx42Njb4888/cePGDahUKnz33Xes4X737p2kMXH8/f3h4eGB9PR01sC7u7vDw8MDH330kdEOrHr16ujevTuaNm2KrVu3SmLECQoKymLAMUalSpXYfsiUlBRWUSpUqGCy1SORI0eOID09He7u7ixegzFvlsKFC7O9irdv35Zsq5IuKSkpbFVdDL5Zt25dbNmyBTVr1pT8+rpkZGRgyJAhLNhl2bJlAQDNmzdHSkqK2T2TdDl8+DDzlgCE4IVyBgnODS9fvoSrqyusrKwwYcIEANCb5JqT0NBQxMTEID4+HrVr19Y7l5CQgOvXr7N4D6ZCo9GgcuXKGDt2bI7fe+jQIbx+/dpsAUtDQkLwzTffoESJEtizZw86dOiQ44Bf17gVHR1tFlduXX777TcEBwejaNGiGDJkiNHP3Lp1C9OnT8fRo0ehUqlQpkwZnDt3DgAk3TobERGBN2/eYOnSpcx74tatWwAEIw6lFDNmzJDsWX369Ck6dOigZ3w8ePBgtm23hYUF6tWrhwEDBmDv3r3w8vIyq/dXRkYGwsLCMGPGDLZdVdRlY2MDT09PDBw4ENWrV//grV4JCQkAhIHzsmXL2AJGTqSlpWHFihU4f/48G+ACgL29PSpUqIBly5ZJtpXqhx9+YD9XqVIlTwYrpfDs2TNs2bIFhBD89NNPkl7rxx9/BAAsWLCAeZYBwphQd+XT2CTJnAQEBDDjuGEsJinJbpxy9OhR3LhxAwcOHMDdu3ezxAZ58uQJLl68mKvnJTeYyjhcpUoVVo6mRoxz069fv2yf76ioqCyeclJz8uRJPe/vwYMHo2fPnmbVkFu0Wi2LI6ZExo0bl+XemsMw8fLlS+aZGxUVBUtLSxYT8n3zCVOPRXU5ePAg++4xY8aw+ZetrS2WLl2K3bt3M6/nkJCQLN4nUiF63uoacHLrfSsVfn5+esYbQFgonDdvHnbu3Il69eph9+7dAICSJUvqfS6/7d9/N+8yh8PhcDgcDofD4XA4HE4BwmSeOFWrVmVbcAICAlhMjt27dyM5OZlZh69duwY3Nze2urZz506kpaUZ3YuYX9RqNTw8PKDRaNC9e3f4+voCAHNrygkHBwfJXCMvXryIQYMGGT3XoUMHVK5cGVWrVkW3bt1QrFgxtg3lzZs3zBPn66+/NmnKuxcvXmDNmjWglOYqi5G4+mljY4PChQtjwoQJ+OmnnySLQxMfH58l+ODSpUtlCTytVqv1rNRixhA7Ozs4OTmhe/fuAIQgj4UKFcqVe76pMIwVlF09UwLp6enYt28fFixYAEtLS/zyyy+KcAN2cnIyGs9A16U8p9hHeaVWrVqoVq1ajisqZ8+eZW6969atY1k4pIyv1KRJEyQkJEClUuUq9pbuqpA5vXDEFY4VK1bAzc0NCxYsgL29PZ4/f45Xr17h7NmzAISAfBs3bkSdOnXw448/omvXrmaJGxAREYE+ffqgbNmyzEPo2LFjLI5Keno6JkyYgLlz50py/aSkpCzBdleuXJmr+DudO3fG9OnTsWvXLqSlpUkeHH316tW4du0aQkJCmKu2uFX34MGDKFeunMmC6efk9aDRaPSe9x07duD+/ft49OiRngeOyLx58yR1546IiNDzotINklgQEIO91q5dG7GxsRg+fDjq168v2fWuXbum5+lz+PBh5oljSJ8+fSTTkRvE8ZaYiEFOEhMTMXnyZNy+fTvbz2g0GgwaNAgTJ07MlTf5+xg2bBg2b96MR48eAUCutiqfPHmSfU7cyqFWqxEREZFlpduU5PTdhglSpCY5ORleXl56XmXe3t5m39aSGyIjI3HixAmoVCpF6vvll1+YZ6wuTk5Okoci8PT01HveevXqxZJ6yOnVr4tuGxkREYFZs2bp9Y/t27fXCy0hFbpbqAD5vXAsLCwQGBioF+MQELys27dvj9evX8Pd3R1r1641Ol/XaDQIDAzMlwaTzbi9vLxYozphwgS9yeOgQYP04tFcu3YNMTExAICPPvoIW7duRbNmzXDjxg1TyQEguIWmp6ejSJEiWLp0KT755JMsn0lISMCaNWuwadMm5gopbguQ6gHy9/fHu3fv2O/t27cHIExGrK2tswyUxdRkuu7Tuq7VpsDa2hpFixbFs2fP8vR3YWFhUKlUWL58OYYMGZIlhbypEDt4XVq2bIlPP/0Ua9asQbt27czWOZw6dSrbczExMSx154oVK2BnZ8cG97NmzZI0ivvNmzexe/duUEqZW3Fe9uabA7GNiImJgZ+fHxYtWgQnJyecPXvW6POpJMR4VVOmTDH5Fr6cJsc3btzATz/9xDrJmJgYFv/p0qVLkhlyrKyscr1VJTEx0egzKjW6aXlfv36NvXv3IjU1Fbt378bIkSPx6tUr1i58++23WLRoEUaMGGGWTG3JyckABDfthw8fYurUqbCzs2NBOcVtCnXq1MGiRYsk1SEaRMQ4Qd99912u2ksLCwv0798fS5cuxYsXLyQ1Gi5ZsoTF7AGArl27YubMmSxOiDkH/+vXr2fZJXLDxYsX0bdvX8kmkGq1Ws8I17VrV4wZMwatW7eGvb29XlB4JSMaeqXcAh0XFwdXV1fW1xQuXFgvg44Y0FrMMiRXhk1AcLkPCgoCIARQlWprRG4xnFBmR2RkJCZOnIh+/frlO65RhQoVMHjwYJYdztfXF506dcox607btm1ZWYkxjSZNmiTZNmxxTrB+/fosWReTk5Mxf/58HD9+HJRStmgnVXp4kTdv3uDKlStQqVTMEJibMZRWq0V6ejqsrKwkbVPF/u327dto0qQJCCH45JNPEBwcLNk1P4RTp05h2LBhWZ69kiVL4uLFi7C1tZXkumJWRN2F11GjRrHkA3Kja0ydOXMmGjdujPDwcBw4cCBLWZlrS6rhopvc26jc3d1Z0HeR6OhotG/fHk5OTrhw4UK226Xu3r2L4cOHs8RFH4pJWpjq1auzmzhx4kRMnjwZHh4eAISBtWH07Ddv3rBghCEhIShdujTatm1rUiPOmzdv2P707777DhkZGWxlpm3btixt8Llz59h+V7FiStWRUkpx+vRp/PzzzwCEidvRo0dZoLTsLL5i/KBt27aBEIJly5aZPFtI0aJFUaZMGURGRuL169fvXaUVjVBiCtGePXtKGvgyu+jwDx48gKurK7y9vdkEQKpGV+S7774DIQRBQUGoUKFCFmNfkyZNcOHCBSxfvhxv375l9S4uLg5r166VLF2vh4cHMjIyUKtWLbRt2zbbzz158gQXLlwAAGYF/uKLLzBs2DBJ4xs9efKExSkQB9WUUsTExGDixIno0qUL+vfvD8B4LCa5EIMGbt++HYQQ9O/f32wpl2NjY9G9e3c8fPhQ77hoBPf398e0adPMoiUn7ty5wzSZEwcHB2zcuBGAsFokxg8DhBg3gwYNYploxNhV5kI0pB49ehQAWFregwcP4ujRo0zrnDlzJNURHR3Nfl68eHGe/97Kygr29vaSZRISDVjTpk2DSqWCp6cnpkyZAkdHR9naAa1Wy1KMly5dGhkZGWjcuDFbIdVlzJgx2LFjB65evYrbt29Lorlz587w8vJi7WdMTAwmTZoESikcHR3RqlUrDBo0KMuA0lyIQYTj4+NZYNrPPvsMTk5OSEpKYln9CCFo0aKFZGmgxdTTummq9+3bx1ZB3717xzKpiJ5A5jDoZkdAQAAyMjJQo0aNLKu5cqAbNPx9JCUloWXLlmz8nB+8vLygVquxd+9evH37Fh07dmSxMj/55JMshjbDYMiAMC6Tql9u0qQJQkNDce3aNT2De0REBH7//XcWmLZ06dJsUSE1NRXVqlXDxo0bJVm8W7JkCQgh6Ny5c46ewVqtlhkKr1y5gkePHmHTpk1Ys2YNvvnmG8kWq//3v/8BEAKZE0JMGkDcVKSkpGTrQbVy5UpJFy327NnDDJfiAv2CBQvMNrZ8H/369WNezqdOnWKL156enrC3t9fzdHxfMHsp6NGjB4uPExwcnG0acikxNnbTarVYvXo1unbtmu1YIDk5GdOmTcu3AQfAPxbm3LwsHSpQAFlevXv3phqNhmo0GqpWq2lGRgb73cfHx+jfiK/FixfTjIwMGhwcTAsVKpTjZ41psv/YmfZZd5EakpKSQlu1akVVKhVVqVSUEMJ+1n1ZW1vTfv360bJly+odzw0uLi5GNYm6DElOTqZFixalKpWKtm7dmiYmJr73GtevX6eWlpbU0tKSqlQq2rlz5w/WZUyTLjt37qQWFhb0448/psuWLaPLli2jp0+fpsnJyXr/w+nTp2mrVq1oq1atqIWFBS1XrpzeZ/Kqydj9M0ZsbCyNjY2lwcHBNDg4mA4YMIASQigASgihM2fOpDNnzszVd4kACMtO1/tIS0ujWq022/MJCQnU399fr175+fm993s/tKwaNGhAVSoVrVatGk1ISKAJCQlZPrN9+3Zqb29v9FlwdHSkoaGhedJEc1lWFy5coCVLlqSEkBxf9vb21N7env7999/v/c6cdOWlXuXEjh07aLFixWixYsUoAFqjRg366tUrs2maP38+JYTQevXqsTIqUaIE+3nKlCm5/q781PX34evrq3cfc4spyurOnTv0zp07etdv0KBBrtrXvGiieSira9eu0QoVKtAKFSpQlUpFp06dSjUaDTuuUqlo8+bNafPmzWlQUBANCgr6YF3vK6sJEybkqV8zZOfOndTe3p7Gx8ebTJMutWrVorVq1aIAqJeX1wdpNMQUdf3Vq1f01atXNDk5mSYlJWX7ubdv39K5c+dSQgjt2bMnzcjIMPq5/JbV48eP6ezZs+ns2bOphYUFtbCwoCqViv1sYWFB586dS+fOnUsPHjyYq/8xv3U9Li6O/vDDD7Ry5cq0cuXKTI/43r9/f71xmIWFRa7a9g8tq82bN7M2YOLEiXTixIl692P48OHs/Pnz5+n58+f1/v7t27c0Ojqa/vzzz2wMm19N2ZGRkUF79uxJAdBZs2bl+e9FTNmu37t3jw4dOpS2bt2atm7dmg4dOpTevHmTTpgwgU6YMIEdd3R0ZGPqM2fOZPmeDymr8PBw6uDgkKVO29nZ0TNnzuiNMT09PbM8A1FRUTn+b/mp62q1mrq4uBidRxBCaO3atemiRYvogwcP2N+8e/eOrl69mrq4uNDo6Og868qprJKTk2mNGjUoIYT6+/sb/Ux8fDwNDAykI0eOzHa85eHhYbS9MkVdF+dtKpWKenp6Mh1ff/21Xjl27NiRvnnzJrffabK6/vbtW9qkSROqUqmYTpVKRceOHUvHjh2b6+/5kLJKSkqi9vb2rDySk5PfO4fSJTExkRJCqJOTk9FxvinGMBkZGTQxMZEmJibSv/76i/1MKaWnTp2iFhYW1NnZmTo7O+d6rJXfepWTfUB89ejRg/bo0YPevXs3X5oozd7mIb6ioqKoVqvN82vfvn25+l+y06WnMf9mIOD+/ftITk6GjY0NS3W2ZMkSAO9Pme3j44Nx48ahRYsW+O6777Bq1SpTSGIZlKZMmcJiqYhujrpW6ylTpqBs2bLo168fs/xLlR7X2toa4eHhePjwIb766qv3eozcu3cPTZs2ZRHdCxUqJKnFs2fPnnj58iW+//57Pbf20qVLw8rKClqtFhkZGXqrXIAQx8ccezfFlXTxvWXLlvDz88OQIUOwf/9+ljHrm2++MUs63PdZzIsUKYKePXuyWFHnz5/H6NGjMXz4cElW/9q2bYurV69CrVazeyRup9JoNBg7diz8/PxgbW3NYlR17NgRAQEBCAoKQlxcHMv8ZWrKly8PW1tbtrd28uTJbKUbENIHr1y5kmXA+eqrrxAZGSl55rP09PQs9fnixYvMQr59+3a2yuzg4IBz587lKpaIqRBd26tVq4Y//vgDAHD8+HG2Grh+/XpMnz5dMi+J3HL16lWUKFECx44dM+t137x5w7KxFS1aFJ9//jnCw8Nx9epVDB06FOvXr5c0jbExXr9+jXXr1uHx48cABM9ArVaL4sWLIzk5GZRSEEJYVqwOHToAEDx2xJ9NyaNHj0Ap/eAtNw0bNoRGo2HbeqWEUoqkpCTJvSlzQ26f82LFimHs2LE4fvw49u3bh6CgIOZpbErKly8Pb29vAELsi9DQUJw4cQJRUVEsRo443lKpVFiyZAnGjh1rch2AkGWqY8eOCA8PZ/UZEO5fnTp1WDzBnTt3ghACYfwNnD59WtI4dlu3bgUgxFESPRTT0tIQGBiI9PR03Lx5k31W9KrSzQgXFRWFp0+fAhDqPZDVhd9U3Lt3D3v37oVKpZJszJlXPv/8c2zevDnLcUMPvsjISDRu3BjPnj2DWq1G8+bN833tL7/8EpcuXWLb3EQSEhLQunVrNG/eHDNnzsSLFy9w6dKlfF8vL1SpUgUhISG4evUqunTpAkBYTXd2dsaePXvg7OycxZO+aNGiGDVqFIoVK4bevXvj3LlzJvOy0Gg0bGxgmEUzOTkZISEhcHNzg0aj0fN+ad++PWs7xo8fj40bN2LJkiUmja8pIt4jQgjLyDhw4EB88cUXuHPnDvOAOXr0KPr37w93d3cWJ84chISEIDQ0VM9LqFWrVmbJELdt2zY21hbH5LklIyODeYd37txZspAJKpWK9cOGbaCfnx8AsH7OXP01pZTN84KDg7Fv374snxGP7du3Dz169ICPj49kHjpBQUH49ttv8/x3Jo1/mBtLT26sUjY2NnT//v10xowZtEaNGrmyMomvVatW0YyMDBobG2syTxyRtLQ05pWQlJSU7Yra//73P2aJXbRoUb4teKZY3Q4ODtaz+I8cOTJfunKradu2bbRXr160V69e1MnJiTo5OVFLS0sKgBYqVIja2NgwTT179sy3pvx6TCQnJ9OuXbsyTY0bN6apqam5+ltI6J0gcuvWLXrr1i2mb9u2bTl+/kPLKj4+nt2bOXPm0Dlz5rCVxCNHjlCVSkVtbGzosWPH9P4uMjKSaTM89z5NNA9lpbuqaYzExEQ6ffp0On36dEoIoT169Hjvd35oWb169YpevnyZurq66q36GVvdFl9z5szJ1f9pirp+9uxZevbsWVqoUCFasmRJWr9+fUoIoVOnTqVpaWnU39+f+vv7U0II9fX1zdV3SlXX09PTafny5amTk1Oe/zY/ZRUYGEhLlixJixQpQosUKUJnz55Nk5OT6blz59gql4eHB1v9yK8mmsuymjdvHluhzW7lVvd3Ozs76ujoKIk3Y3R0NLWysqIqlYqGhYXltggYsbGx1NHRkdaqVctkmgxp0KABbdCgAevnmzVrlqf7ZQxztOuG7Nq1ixJC6OjRo42el6oPTEtLo/Hx8XoeVxYWFrR27drv/dsPqetHjhzJ4nXj4eFBPTw8aHR0NE1LS6MrV67Ua0tdXFyoi4sLTUtLy9X/9KFlVaVKlfd6e+b29ebNGz0PAVPfv8aNG1MAtH///nn+W13kqOuUCu0cIYRu2LAhy7n8lJVaraaDBw/O0m5Cx1vC2PHIyMgcv9cUY5ikpCRaqlQpWqpUKWpjY5Ojh43I6dOnKSEkz17OOZXVu3fvWD0NDw9nxxMTE+mSJUvYuRo1atDw8HD2evfuHU1OTqZDhw6lhBBaoUIFo3MiqT2cDdmxYwft2LEj9fX1pffv38/2c6ao66Lnq729fZYx34ULF/KsPa9llZKSQp2cnPLsSS1y7949dn9jYmLypInmsayyo2fPntTCwoJeunSJXrp0Kdd/J2W98vPzoz169DBqO8jJKyc/njgVKlSgarU6i6dNVFQU3bhxIw0MDMxyztfXl1pYWCjLEwcQrL/5CW4kCjI1uQnOmZGRgZSUFHZ9JawCJiQkZAl2aa79fu7u7ln22IaGhiI1NRV2dnaIiIhgFnNxtUpOrK2tsW7dOhY3JDQ0FL///nu22Sj+rTg4OOD777/H4sWL2V7N/v3746OPPmK/d+/ePctqvxj01MbGBk2bNjWppsTERLZS8L4o/7a2tiyY6IIFCyTLDufv74/Zs2cjJiYmT3u0AwMD3+tZaCpE78H09HSMHj0af/31F65evYqPPvoIVlZWetmffHx8MGLECNm8ce7evYvY2FjY29sjISEBACRZ2RN58eIFfv31V8ycORMVKlTA7NmzAfzjYdm0aVNERUXB3t4ev/zyCztuitXi3PC+wI3NmjVDzZo1WaaqihUrSuZxlpGRwTxoPmQV+NixY4iLi2Orz1IwYcIEAELGmQ0bNuDx48d49eqV5F54IrGxsYiOjkblypVZ7JYPoXPnzrC1tYWfn5/JPIpzg5WVFRwcHDB58mQWO2769Ol48OABBg4cyLxTTMGmTZvg4eEBQghsbW0RHBycJcvU5cuX8cMPP+h54IheaUlJSZJ6xu3evdtoDDFdbGxsUKNGDb1jVlZWGDlyJDp37oyXL1+ibNmykiYhiIiIwOXLlwEoO4OkMbRaLfbu3StZUNEqVapg48aNes/QqVOnEBQUBEIIXrx4gd9++42dE70ozBFv5f79+yz+TYUKFfIUFNvb25vFRzMlmzZtYsk0Ro4cqZftbP369XrjgkePHmH27NksLmf//v1zlXlSavr164evv/4a/fv3x/r163Hy5ElUrFjR5NdJTU1lnhNi4GURd3d3FkRfSu7evYtHjx7B0dERXl5eefrb27dvo1mzZrC0tMTmzZtRoUIFiVRmjxgHqly5coqY/4mMHDkSbdq0Yb+Lnjj79u2Ds7Mz7t69a/I59OPHj1GnTh0UKlSIjeeSk5OxdetWaLVa1sYD/8QmXLZsmUm9mpWX643D4XA4HA6Hw+FwOBwOh5MF5aSAkRG1Wo2DBw8yS/7gwYNlViRoEiPKixknxIxfcqBrcdWNuK1raZSTcuXKwdHREYCQavvFixeSX/PVq1cIDw/PknJSTnr37q1n6a1bty5sbGxYefzyyy96n09LS8PMmTMBCNlzTL2/Vs7MH4YkJiaiZ8+eOHPmDNLT09G9e3c0bNiQxWho27YtRo4ciW3bthn9e7Vajbi4OMmzHGVkZLAYF5aWljhw4ADCw8Ph5OSEJk2aAPhnH3yzZs1w7tw5nD9/Hp06dZJUV05QSvHy5Uu2Al69enXJrrVixQr8+OOP6NOnD3bt2mX0MydOnAAgeDKJZWYuhg8fjrCwMCQmJmY55+DggF27drG2SmpKlCgBR0dHPH361KienHj+/DnL7GXqDGgJCQm4c+cO6tSpg969ewMQMsAcPHgQkZGRWLRoERYsWGDSaxoiehKMGjUKz549w/79++Hm5vbB33flyhWWvloOypQpw2LgbNmyBXfu3MHOnTtN4onz7NkzAEKKekII6tati8DAQL14MgAwduxYbNmyhX1m4sSJ2LVrF8vUNmnSJMyfPx9lypTJtyZj1K5dG3fu3GF6Re7fv88yNi5cuBCjRo3K9jvM4dHYuXNnZGRkoFq1amjVqpXk1zNEzC569epVvYx+2XHo0CEAwI0bN5CUlMTiUkiFoQd9165d0bVrVwCCNwWlVPb0wrlFHDvntf3NCUtLS1SuXBkRERE4f/48G9eJHkrVq1fH1q1bYWlpifT0dFy9ehWAEItDnFf07ds3z54gOXHp0iVUr179g5+f4sWLY+XKlahUqRLi4uJM7okTFhaGBQsW4NGjR1nOde/eHRs3bpR8vJqYmIiRI0cCADZu3IhixYrl6u/2798PQIjj9fr1a0yYMAEDBw6UTGdOeHt7gxDCPE+URNWqVfXahTVr1rAYOcHBwZLsZklKSkJSUhKWLVumd7xfv37M4zM9PZ2NZ2JjY016/f+8EUej0bCtJuJg0hxBenPi3bt3eoMMcVuV3LqMYZIUadmg1Wpx9uxZxMbGonTp0mywo1LpO5BlZGQgLCwM58+fZ8cOHTqU40DNFISHh6NNmzYspeonn3yS5TO6jbY5qFevHoKDg/HNN98AEFKw6w4elixZgoYNGzJD3G+//YbQ0FDY29tLEnAyL1s40tPT9YxMppp8p6amAhACYV+7dg02Njbo27cvVq9eDQBsUFO5cmXExcVBq9WiUKFCKFu2LPbs2QNXV1cAQsDa5cuXSz65jIiIYJOQqlWromrVqoiJicHChQuZ8Ua8f56enjh37hx+/fVX2Yw4YWFhIISgXLlykgUB1UUMNN2yZUv2s4hGo8GiRYvg5+cHS0tLdOrUyexpTd3d3dG6dWuEhYUBANtm7ODggMuXL5vNgAMIRpzTp0/D2dkZTZs2zbUb7/PnzzF+/HjExcXB3d3d5KlWvby8sHLlSlhbW6NRo0YAhID0o0ePxowZM7Bz505MmjRJ0i1V4v/07NkzODg4oH379vn6vmfPnkGr1Uqy5ZJSihEjRrAg+eXKlcPKlStBqRBUODY2FmPGjGFJEAz7yPwitqGJiYmwtbVlBpykpCTcu3eP/c/iYliLFi1w+PBh2NraolWrViwQ5+bNm1GjRg14enpKlkrXxsYmS18cFxfHfu7evbsk180NDx48AAA2Zvj555/NutCRnp6O69evo2nTptBoNB/0HWKdE5EqsGp2FC5cGCVLljTrNY0RHx+PmJiY926pkmLcbm1tjaCgIFSsWBE3btxgQbpFvvjiC5w+fRqBgYEIDAxEREQEO1e+fHn0798fY8eONenWxqZNm8LV1RVOTk6YPHlyno0wI0eOZFvm5s+fz7aUm4LU1FQsWLAA+/bty9I27tixw2yhF54+fcoCPufGeJuRkYH169ezRDMpKSnw8vKSJaW3yK5du0AIYVt3OcaZMWMG+/n169dYv369JNf5zxtxUlJSmPVaHNS9L3aH1IwdO5ZZzi9evJhl/7aS0B0cmZo7d+7o7XEUV0nt7OxQpkwZNhjTXV0QMYeVWIzjJA5gRe8JEa1Wi99++03PuFSkSJEs8YZMTbNmzVgWjilTpiAoKAj370eFlcYAAA4cSURBVN8HIMRKMJzUtmjRAn5+flmyQpibQ4cOYfr06QCAkiVLsgx3+SU9PR0AcP36dRBC8Mknn8DBwQF2dnZZykI8P2/ePLbSIRomrly5guPHj8Pb21vSeAmnTp1iP6vVaqjVarRq1cqokc3V1RXNmjXDwYMH8eDBA72MX+Zi0qRJAISVNHO0nU+ePAEgDPqyM5BWrFgRY8aMYdl8zE3x4sVZ9hAxLsjly5fNkjXPkHLlyqFu3br4448/4OnpCQBYvXp1jvdqzpw52L59O8qWLYtNmzaZfKLZpk0bXL58GZcvX8bp06cBgL0DwMOHD/Hw4UNJjTjiCrmXlxcWLFiA5cuXM8+jvBIXF4fJkycD+CcLpinZuHEjNm/ezNqruLg49O3bV29CLWZZEX82Jb///jv7XkdHR1y9ehVPnjzBwoULcf/+fXa9cuXKYcWKFfj6669ZbMEyZcqgbt26AARPjnHjxuHPP//EvXv34O7ujmHDhplUqzFmzJgBSilWr15tViOqLmlpaWyyqNFoMGTIELi4uJhVQ5cuXbKMlfKKbt0qU6YM+vfvn19ZBYYaNWqwxaXz58+jQ4cOOH78eI6GnEuXLoFSygyspsLJyQn79+/HxIkTce/ePb1zAQEBCAgIyGJwc3d3R9euXdGrVy+TagGEnQIqlQpr167Fhg0bWL8nxsbSfT9//jwzRBw7dkzv3CeffGJ0QfRDSU5OhoeHBw4cOMDiJwFC3R02bJhZjbritXv37v3eMeTLly/Rq1cvhISEsMyS69evlySDZUFBrVbn2ptGrVbrLeTrziWlplevXnrzKcPsfqZEdiNOt27dMHr0aGi1Wvj4+MgtR1a0Wi0OHz7MJv19+/ZF3bp1zb6SnBukCEJtiO6gHgBzyRYRJ/mGHVXDhg3N0jCLwfTEtMopKSl6qy5bt27F0KFD9bSJQVilRlwdW716NZKSkrBw4ULMmzcPJUqUYIFEAWDEiBEoVqyYrNueNBoNW20QV2c3b94s2bYl0TAiIrr2t2rVCuPGjUOlSpX00guPHj0agBCA8ubNm7h3756k24UM74WlpSXmzp3LJkK62NnZ4ciRI/j555/h5+eXJRi6OYiPjwchxCyTMUAI5DhlyhTs2rWLpbrUpU2bNtixY4esAepPnjzJDJIWFhZo1qxZnoJgmpJixYph6tSp6NOnD1sNcnNzQ7t27fS2xsbExLAAmRs3bmTeHlIYLN3c3ODq6op3797B19cXgBDM+86dOwCEwNjilg+pENvlKVOm4Mcff2QrZx9iyAkNDUVMTAwAMM8iU9KtWzfMnTs3T67YTZo0Yd6G+aVWrVoAhL723r176NKlC+t3bW1tMXz4cADCtjtjW6XOnj0LQNjuFBERgc2bN8PW1hY//fSTSfS9jwsXLqBIkSLo3r27LOMpSikePnyIK1eusGPz58/Xe/7MgeH2tw9BvO8ODg4m3xqQVx2AMG6uXbt2voKS5wVxC2/jxo2hVqtRpUoVLF68GN27d0fJkiXZlsrY2Fj8+OOP+O2330AIYYsdpkKlUsHNzQ1NmjRhC4lr167V+0y3bt3QpUsXDBgwAIAwljC1l56IeO2BAweCEILGjRsznVqtVu+9adOmegZn8dxPP/2EXr16mXQr1dOnT7Fz584sx11cXMzu0XL8+HEAQv+W032IiIhAx44dERUVBScnJ4SGhgIQjORycu7cOfazOYJAG6Lr6d2jRw+0adOGGWp69OgBAEbTjksR1DgnxEQQonHy1q1bkl1LNiOOuAq2YsUKaLVahISEYN26dWbXER4ezjoDuS2cx44dY8aH6tWrY+nSpZK5HOeHTz/9FJ9//jkAwS1YNwORKRkwYEAWT5acEB/iLVu2mNUoIW59admyJWbPno23b99i//792LVrl15D7eTkxLJ6mRNbW1t4e3vL4pWg1WqRmpoKS0tLWFlZMRfu9PR0xMfH448//sD+/fvh7++PTz/9FFu2bAEAk8YZErMvbN68Gd7e3oiJiUG9evXQuHFjEEJYrKnsGnlzZpoBkCWzyubNm3PcWla0aFGcPXsW+/fvR/HixVGhQgW20maujFWUUrO51xYpUgS1a9dG7dq1zTYJzAuxsbF6g8NmzZrpeVfJQceOHbFgwQJmoOjcuTMqVaqERo0aoX79+li7di2ePn2KV69eAQAcHR0RGhoq6T0VsyqJRhwfHx88ffoUgNBmmSs7VZEiRXD27FlMmTIFS5YswVdffZWrVTvRC7V58+bMO+zMmTNGja35pUyZMrhy5QobzPv6+uKPP/7I9vMrV66Eu7u7ybZLiNs4p0+fDh8fHxBC4Obmhnnz5qFs2bLvNQ6IBtULFy7A1dUVhBD4+fllyWwlBVFRUUhLS8Ovv/5q8m2BuSU6OhpVqlRhv0+ePFkWLVu3bkXLli0xevRotkVOl969eyMgICDbvxdDDohjCrlwcXFhxjiVSoUmTZpImvVMF/G+hYeHw83NDefOncOYMWMwZswYVK1aFc+fPwcAlsXK1tYWO3bskCzDX8mSJdlihrFFDXMjzu2Mbd2NjIzEkiVL9BZeza1ZzJwph0Eku6x5ycnJAICjR49ix44dOHToENLS0uDi4oLDhw/LbrwRiY+Ph1arhYODAzPsmxNKKXr27AlAMNboGmyMGW8A8xtwADCPa9G2sGfPHmb0EndFmAqzGnG8vb1x69Yt1KtXDz/88AMAYdU5Ojoaa9euZdsezMmFCxdYY1KzZk2zXx/450aLweMAIZVco0aN8NdffykuFk7RokXZaltkZCSaNGkCNzc3zJ0716TXsbe3x+nTp/H333/jypUrbIsQIEysRYOAuNIgdpLmKq+GDRuiS5cuzEPo6tWr6Ny5Mzsv1itxBf7cuXOypASUkytXrqBx48ZwdnZG7dq18ffffwMASx+uUqlgbW3N0mTresCYCnHbyODBg9G3b19kZGTAysoq14Y+Mb7Q1atXMX78eMnjvvTp04d5maWkpORqC87AgQOxd+9ezJo1C1ZWVln2yEuFuL+bEMLu7X+d7du34/r162z7m6EHoRwUKVIEo0ePRlpaGgDBmykqKgr37t3D1q1bQSmFlZUVM4RPnDjR7HveLSwsZNlnTwhB06ZN4ePjg1atWqF9+/b44YcfmCeVuD3q9evXOHDgAIKDg3H58mXmeSOuvM+dO5cZhqWgfPny7P6I7+ZCnCDPmzcvX6vXpUuXZrGizMW+fftgZWWV75hH+UGMaSEudskZ02LYsGE5ek3u3r3bjGo+jN69e+O7776TVYOdnR3OnDmDNWvWwN/fH2FhYVCr1Wy8N3bsWHTr1g0lS5ZUzCRcbj777DPZDU3iPKtZs2ayaTh8+DAOHDgAa2trnDhxgmnSjV20bNkyjBo1SlEL+Xfv3oVKpcLs2bNlm5fqBi4W+1oxlbgufn5+Zo1HmhNFixZFvXr1ABRgI06NGjUwc+bMLNtwrl69Cl9fXxZ9+7+M4cMaHR1t0nzypkSMxN26dWvcunULb968wfjx400+CVepVHB2ds4ycZZyj2Fusba2xp49e7BixQoAwsqsYYTy8ePHswGb0oxx5kD0lpg6dSqsra2Zy7OzszPatGmDCRMmmDWOy4fcgxEjRui9S03NmjWzBOx9H25ubibfc58bdF1qTd05FXTEbCriyp/c2Nrasi1DM2bMQEREBJKTk7Fu3Tp07NgRNWrUkCVmj1Jo3rw5goKC0L59eyxfvpx54InelFqt1mh/PGfOHHTq1EmxW5//q4ir24cPH8bSpUtlDYZ75MgRWFtbM6O3krI2FkRsbGwwZswYAIJ3UWJiIjQajdm3pwE5x2bjyM9nn332wUG8TY2vry/zPBXp0KGDIuYzuUGMjdaxY0eZlQiYI6zHh7B582bUr1+fzenT0tIk23oqzeZIDofD4XA4HA6Hw+FwOByOSTGb2frPP//EqFGj4ObmhsePH7MUi0uXLjW6N/e/hLiCvmPHDnbs888/x4IFC1g8D6Xx1VdfARDSn48aNQr9+/eXZCuM0ilUqJBewLqCYlE3F9bW1pg8eTLL3ML5d2FtbY0+ffrg1q1b2LBhg9xyFMGECRP0gocrFXG715o1a2RWogwIIWjbtq0sHm0c0yOOnQwTJMhBXj0rOTljZWWFpUuXAgDq16+PAQMGYPbs2bJkZ+Rw/gu8e/dO0mzE/yZ++eUX1KxZE+PGjQMgJJg5c+aMJNcyq+/hunXrZAlenBNiwFBfX1/Z9q2K7tqjRo3C/Pnz4fH/9u7gxmkoisLwcQGpgSbYpRE34FF6yCopIXXRADXADrE2q5FgGKEXQeJj9H0V/Hq5qxvb7+Ul1+v1n9wk8Gin0+lpr5kAfV5v6wCAZ5rnOfM8b50B/7XD4ZDz+Zzb7WZZOuBZf+ZtfsX41o7H4+bfnXl9j/5yueRyuWzaAgAAAEmyLEuWZdk6g59M93wYaJqmb0k+Py7njz6s6/rboynTNH1J8j3J1+cnvd+U1J5VY9OWv1/irEbtbdbrzsqsv6vxrMz6uMamxKyPMuvjGpsSsz7KrI9rbErM+iizPm5XZ/XWvUucT+u6fvyrrAdo7NI0rrGrsSnp7GpsSjq7NI1r7GpsSjq7NI1r7GpsSjq7NI1r7GpsSjq7NI1r7GpsSjq7GpvecjsVAAAAwA5Y4gAAAADswL1LnNY7ZBu7NI1r7GpsSjq7GpuSzi5N4xq7GpuSzi5N4xq7GpuSzi5N4xq7GpuSzi5N4xq7GpuSzq7Gpl/c9U0cAAAAALbhdSoAAACAHbDEAQAAANgBSxwAAACAHbDEAQAAANgBSxwAAACAHfgBFzDZQP7S9wwAAAAASUVORK5CYII=\n",
            "text/plain": "<Figure size 1440x72 with 30 Axes>"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "We can also send raw HTTP request to the service."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import requests\nimport json\n\n# send a random row from the test set to score\nrandom_index = np.random.randint(0, len(X_test)-1)\ninput_data = \"{\\\"data\\\": [\" + str(list(X_test[random_index])) + \"]}\"\n\nheaders = {'Content-Type':'application/json'}\n\nresp = requests.post(service.scoring_uri, input_data, headers=headers)\n\nprint(\"POST to url\", service.scoring_uri)\n#print(\"input data:\", input_data)\nprint(\"label:\", y_test[random_index])\nprint(\"prediction:\", resp.text)",
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "stream",
          "text": "POST to url http://52.142.234.83:80/score\nlabel: 0\nprediction: [0]\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Let's look at the workspace after the web service was deployed. You should see \n* a registered model named 'model' and with the id 'model:1'\n* an image called 'tf-mnist' and with a docker image location pointing to your workspace's Azure Container Registry (ACR)  \n* a webservice called 'tf-mnist' with some scoring URL"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "models = ws.models\nfor name, model in models.items():\n    print(\"Model: {}, ID: {}\".format(name, model.id))\n    \nimages = ws.images\nfor name, image in images.items():\n    print(\"Image: {}, location: {}\".format(name, image.image_location))\n    \nwebservices = ws.webservices\nfor name, webservice in webservices.items():\n    print(\"Webservice: {}, scoring URI: {}\".format(name, webservice.scoring_uri))",
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "stream",
          "text": "Model: estimation_de_pi, ID: estimation_de_pi:3\nModel: sklearn_mnist, ID: sklearn_mnist:2\nModel: modelevoiture, ID: modelevoiture:1\nModel: AutoMLf715298d5best, ID: AutoMLf715298d5best:1\nModel: pi_estimate, ID: pi_estimate:2\nModel: estimatation_de_pi, ID: estimatation_de_pi:1\nModel: pytorch-hymenoptera, ID: pytorch-hymenoptera:1\nModel: best_model, ID: best_model:1\nModel: AutoML6ab8e3002best, ID: AutoML6ab8e3002best:1\nModel: tf-dnn-mnist, ID: tf-dnn-mnist:1\nModel: usedcarsmodel, ID: usedcarsmodel:13\nImage: area-calculator, location: amlsworkspace0456663123.azurecr.io/area-calculator:2\nImage: usedcarsmlservice02, location: amlsworkspace0456663123.azurecr.io/usedcarsmlservice02:1\nImage: area-calculator3, location: amlsworkspace0456663123.azurecr.io/area-calculator3:1\nImage: usedcarsmlservice01, location: amlsworkspace0456663123.azurecr.io/usedcarsmlservice01:1\nImage: modelevoiture, location: amlsworkspace0456663123.azurecr.io/modelevoiture:1\nImage: usedcarsaksservice, location: amlsworkspace0456663123.azurecr.io/usedcarsaksservice:1\nImage: areacalculator, location: amlsworkspace0456663123.azurecr.io/areacalculator:1\nImage: usedcarsmlservice, location: amlsworkspace0456663123.azurecr.io/usedcarsmlservice:1\nImage: sklearn-mnist-svc, location: amlsworkspace0456663123.azurecr.io/sklearn-mnist-svc:2\nImage: usedcarsaksservice2, location: amlsworkspace0456663123.azurecr.io/usedcarsaksservice2:1\nImage: tf-mnist-svc, location: amlsworkspace0456663123.azurecr.io/tf-mnist-svc:1\nImage: areacalculator2, location: amlsworkspace0456663123.azurecr.io/areacalculator2:1\nImage: monservicevoitureaci, location: amlsworkspace0456663123.azurecr.io/monservicevoitureaci:1\nImage: aci-hymenoptera, location: amlsworkspace0456663123.azurecr.io/aci-hymenoptera:1\nImage: monservicevoitureaks, location: amlsworkspace0456663123.azurecr.io/monservicevoitureaks:1\nImage: usedcarsaksservice3, location: amlsworkspace0456663123.azurecr.io/usedcarsaksservice3:1\nWebservice: tf-mnist-svc, scoring URI: http://52.142.234.83:80/score\nWebservice: areacalculator2, scoring URI: None\nWebservice: areacalculator, scoring URI: None\nWebservice: area-calculator, scoring URI: None\nWebservice: monservicevoitureaks, scoring URI: http://13.66.133.170/api/v1/service/monservicevoitureaks/score\nWebservice: monservicevoitureaci, scoring URI: http://40.68.19.44:80/score\nWebservice: usedcarsaksservice3, scoring URI: http://13.66.133.170/api/v1/service/usedcarsaksservice3/score\nWebservice: usedcarsmlservice, scoring URI: http://13.80.108.189:80/score\nWebservice: modelevoiture, scoring URI: http://40.68.77.153:80/score\nWebservice: usedcarsaksservice2, scoring URI: http://52.183.40.79/api/v1/service/usedcarsaksservice2/score\nWebservice: usedcarsmlservice02, scoring URI: http://51.144.32.255:80/score\nWebservice: usedcarsaksservice, scoring URI: http://52.183.40.79/api/v1/service/usedcarsaksservice/score\nWebservice: usedcarsmlservice01, scoring URI: http://13.95.6.243:80/score\n",
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Clean up\nYou can delete the ACI deployment with a simple delete API call."
    },
    {
      "metadata": {
        "trusted": false
      },
      "cell_type": "code",
      "source": "service.delete()",
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "minxia"
      }
    ],
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    },
    "msauthor": "minxia"
  },
  "nbformat": 4,
  "nbformat_minor": 2
}