nb/prtf-boltzmann-1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boltzmann portfolios\n",
    "\n",
    "### Part 1: Single decision relationship to Markowitz\n",
    "\n",
    "We develop an alternative to the traditional offline mean-variance\n",
    "framework (\"*Markowitz*\" portfolios) called ***Boltzmann*** portfolios\n",
    "which addresses growth of wealth and its uncertainties from the\n",
    "standpoint of cross-entropy and optimal sequential decisions.\n",
    "The improved result is a faster online algorithm which is more robust.\n",
    "\n",
    "Boltzmann portfolios rely on geometric mean rates since they \n",
    "are optimal under logarithmic utility in the mean-variance framework. \n",
    "Accuracy on risk estimation has been improved by using our research on\n",
    "Gaussian mixtures presented in https://git.io/gmix --\n",
    "specifically, our function gemrat() computes the geometric mean\n",
    "rate by taking into account the fourth central moment, kurtosis.\n",
    "This is crucial for *leptokurtotic* (\"fat-tailed\") assets.\n",
    "\n",
    "After the geometric mean rates of the underlying assets\n",
    "have been estimated, we use the covariance matrix to inform us\n",
    "of possible inter-correlations we can exploit to\n",
    "minimize the variance of the Boltzmann portfolio.\n",
    "Such minimization in fact maximizes the portfolio's terminal value\n",
    "over multiple periods.\n",
    "\n",
    "Markowitz portfolios are constructed in the arithmetic mean-variance\n",
    "framework for a *single-period*. They are fragile to changing\n",
    "market conditions out-of-sample, much like elegant battle strategies\n",
    "which crumble under harsh war conditions.\n",
    "In contrast, Boltzmann portfolios are designed to be\n",
    "*adaptive* over multiple periods to maximize final wealth.\n",
    "Techniques have been borrowed from Bayesian and reinforcement learning.\n",
    "\n",
    "Multi-period process will be covered in Part 2.\n",
    "Sequential decision-making using Boltzmann porfolio weights\n",
    "will be covered in another notebook, Part 3.\n",
    "The mathematical proofs will be given in Part 4."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Dependencies:*\n",
    "\n",
    "- Repository: https://github.com/rsvp/fecon235\n",
    "     \n",
    "*CHANGE LOG*\n",
    "\n",
    "    2017-07-08  Add conceptual explanations.\n",
    "    2017-06-28  Refactor subroutines to lib/ys_prtf_boltzmann.py\n",
    "    2017-06-27  Rewrite relying more arrays than lists, for speed later.\n",
    "    2017-06-26  Rough draft, test subroutines."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "\n",
    "Traditional portfolio allocation (see References) develop\n",
    "what is generally known as the \"**efficient frontier**\"\n",
    "pictured here:\n",
    "\n",
    "![Efficient mean-variance frontier](http://webpage.pace.edu/pviswanath/notes/investments/gif/assetalloc77.gif)\n",
    "\n",
    "A student in finance may first believe the frontier's usefulness\n",
    "in portfolio management, but later as a professional will perhaps\n",
    "voice a **critique** along these lines:\n",
    "\n",
    "> What I now find problematic is the theoretical concept that one\n",
    "needs to only pick a point along the top of the bullet to achieve\n",
    "the optimal balance between risk and reward.\n",
    "Yeah, like it's so easy to just slide your finger along the curve\n",
    "and select at your leisure what you'd like to earn on your money.\n",
    "The focus on past prices that make up these curves\n",
    "and the assumption that the covariance structure is stationary\n",
    "in perpetuity is so wrong in practice.\n",
    "What is rarely talked about is that the future will\n",
    "NOT fall along that line. It is nice to know what you\n",
    "could have earned, but what you will earn surely will not be so\n",
    "easily cherry-picked off of some rear-view historic curve.\n",
    "--(edited Market Tech comment on 2015-04-10)\n",
    "\n",
    "Please pay particular attention to the apex of the hyperbolic curve \n",
    "(the \"bullet\") which indicates the point of **global minimum variance**.\n",
    "We shall borrow that, and ignore the remaining CAPM machinery since we will\n",
    "not be interested here in asset pricing relative to the market portfolio."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reframing the issues\n",
    "\n",
    "One virtually has no control over how the assets perform and interact.\n",
    "Only the portfolio allocation over time is in our decision set.\n",
    "\n",
    "Let's recast the underlying assets as *agents* which supposedly will\n",
    "help increase our wealth. Our task will be to select the *expert(s)*\n",
    "among the agents and to allocate portions of our current wealth.\n",
    "\n",
    "To discriminate among the agents we need their performance metrics.\n",
    "Since our objective is to maximize future wealth, the\n",
    "optimal metric is the geometric mean rate of each agent.\n",
    "From our research we know how to include risks, including\n",
    "leptokurtotic events (\"fat tails\"), into that single metric.\n",
    "\n",
    "There is evidence that the performance of some agents are\n",
    "inter-correlated. Therefore, rather than select a single expert,\n",
    "we choose to diversify our bets among a few agents,\n",
    "and call that our \"portfolio.\"\n",
    "To maximize the geometric mean rate of the portfolio,\n",
    "the second order condition is to minimize its variance.\n",
    "That problem is easily solved by borrowing the weights\n",
    "of what is known as the \"Markowitz global minimum variance portfolio.\"\n",
    "\n",
    "Those weights depend on the covariance structure of the\n",
    "agents' performance which is unfortunately not stable over time.\n",
    "There may be some information which can be exploited\n",
    "to tilt our bets favorably.\n",
    "\n",
    "```\n",
    "    prices ---> cov ---> globalw\n",
    "      |                    |\n",
    "      |                  trimit  <-- floor\n",
    "      |                  renormalize\n",
    "      |                    |\n",
    "      v                    v\n",
    "      |                    |\n",
    "    gemrat              weights\n",
    "      |                    |\n",
    "      |________scores______|\n",
    "                 |\n",
    "                 |                   Boltzmann\n",
    "      temp --> softmax --> probs --> pweights\n",
    "```\n",
    "\n",
    "The Markowitz weights may suggest that we bet against the\n",
    "consistently poor performance of some agents.\n",
    "We shall generally regard the weights as advisory,\n",
    "taking what suits us and renormalizing.\n",
    "\n",
    "To summarize the information set so far, we cast\n",
    "the agents in a game, each with some score.\n",
    "When the game consists of multiple rounds,\n",
    "we can use tools from reinforcement learning\n",
    "to help us make the best sequential decisions.\n",
    "\n",
    "The softmax function is fed the scores to compute\n",
    "the probability of a particular agent being the expert.\n",
    "This function takes temperature as a diffusion parameter,\n",
    "that is, an optimal way to diversify our bets across possible experts.\n",
    "The theory here is due to Ludwig **Boltzmann** and his work on\n",
    "statistical mechanics and entropy.\n",
    "But the temperature setting can also be seen as a Bayesian\n",
    "way to express the overall uncertainty involved with\n",
    "estimating the various measures.\n",
    "\n",
    "Finally, those probabilities are combined with our\n",
    "renormalized weights to arrive at \"pweights,\"\n",
    "our portfolio weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from fecon235.fecon235 import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ::  Python 2.7.13\n",
      " ::  IPython 5.1.0\n",
      " ::  jupyter_core 4.2.1\n",
      " ::  notebook 4.1.0\n",
      " ::  matplotlib 1.5.1\n",
      " ::  numpy 1.11.0\n",
      " ::  scipy 0.17.0\n",
      " ::  sympy 1.0\n",
      " ::  pandas 0.19.2\n",
      " ::  pandas_datareader 0.2.1\n",
      " ::  Repository: fecon235 v5.17.0603 devPrtf\n",
      " ::  Timestamp: 2017-07-09T01:48:51Z\n",
      " ::  $pwd: /media/yaya/virt15h/virt/dbx/Dropbox/ipy/fecon235/nb\n"
     ]
    }
   ],
   "source": [
    "#  PREAMBLE-p6.15.1223d :: Settings and system details\n",
    "from __future__ import absolute_import, print_function, division\n",
    "system.specs()\n",
    "pwd = system.getpwd()   # present working directory as variable.\n",
    "print(\" ::  $pwd:\", pwd)\n",
    "#  If a module is modified, automatically reload it:\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "#       Use 0 to disable this feature.\n",
    "\n",
    "#  Notebook DISPLAY options:\n",
    "#      Represent pandas DataFrames as text; not HTML representation:\n",
    "import pandas as pd\n",
    "pd.set_option( 'display.notebook_repr_html', False )\n",
    "from IPython.display import HTML # useful for snippets\n",
    "#  e.g. HTML('<iframe src=http://en.mobile.wikipedia.org/?useformat=mobile width=700 height=350></iframe>')\n",
    "from IPython.display import Image \n",
    "#  e.g. Image(filename='holt-winters-equations.png', embed=True) # url= also works\n",
    "from IPython.display import YouTubeVideo\n",
    "#  e.g. YouTubeVideo('1j_HxD4iLn8', start='43', width=600, height=400)\n",
    "from IPython.core import page\n",
    "get_ipython().set_hook('show_in_pager', page.as_hook(page.display_page), 0)\n",
    "#  Or equivalently in config file: \"InteractiveShell.display_page = True\", \n",
    "#  which will display results in secondary notebook pager frame in a cell.\n",
    "\n",
    "#  Generate PLOTS inside notebook, \"inline\" generates static png:\n",
    "%matplotlib inline   \n",
    "#          \"notebook\" argument allows interactive zoom and resize."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be stepping through the algorithm with real data,\n",
    "so that the code later will not look like a black box.\n",
    "The numerical output along the way can guide and test\n",
    "the construction of a Boltzmann portfolio.\n",
    "\n",
    "## Download data and construct a dataframe\n",
    "\n",
    "We retrieve the following data of daily frequency\n",
    "representing equities worldwide and gold by five ETF securities: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'America': 's4spy',\n",
       " 'Emerging': 's4eem',\n",
       " 'Europe': 's4ezu',\n",
       " 'Gold': 's4gld',\n",
       " 'Japan': 's4ewj'}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Convenient dictionary set in fecon235.py,\n",
    "#  where the keys are world regions,\n",
    "#  and the values fecon235 stock slang:\n",
    "world4d\n",
    "\n",
    "#  Gold is included as a proxy \"safe-haven\"\n",
    "#  but serves also to test the covariance structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#  Or manually specify your own dictionary here:\n",
    "prices_dic = world4d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ::  Retrieved from Google Finance: SPY\n",
      " ::  Retrieved from Google Finance: EEM\n",
      " ::  Retrieved from Google Finance: EZU\n",
      " ::  Retrieved from Google Finance: GLD\n",
      " ::  Retrieved from Google Finance: EWJ\n"
     ]
    }
   ],
   "source": [
    "#  Download data into a dataframe, alphabetically by key:\n",
    "prices = groupget( prices_dic, maxi=3650 )\n",
    "#  ... about ten years worth."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ========= Specify DATES for  [start:end] =========\n",
    "\n",
    "We can analyze different epochs of history\n",
    "by executing **\"Cell\" > \"Run All Below\"** from the Jupyter menu,\n",
    "*without downloading the data again*.\n",
    "\n",
    "**Include CONSTANTS for the remainder of this notebook.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#  Set the start and end dates:\n",
    "start = '2011-01-01'\n",
    "end = '2017-06-26'\n",
    "\n",
    "#  CONSTANTS\n",
    "MIN_weight = 0.01\n",
    "TEMPERATURE = 55"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary statistics: \"prices\" dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "           America     Emerging       Europe         Gold        Japan\n",
      "count  1630.000000  1630.000000  1630.000000  1630.000000  1630.000000\n",
      "mean    177.792006    40.000503    35.414362   132.703399    44.972448\n",
      "std      35.941680     4.116889     4.192482    20.404781     5.006797\n",
      "min     109.930000    28.250000    24.990000   100.500000    34.600000\n",
      "25%     140.490000    37.542500    32.770000   117.190000    40.320000\n",
      "50%     186.300000    40.295000    35.465000   125.515000    46.000000\n",
      "75%     207.995000    42.667500    38.790000   152.687500    48.595000\n",
      "max     244.660000    50.210000    44.190000   184.590000    54.900000\n",
      "\n",
      " ::  Index on min:\n",
      "America    2011-10-03\n",
      "Emerging   2016-01-20\n",
      "Europe     2012-07-24\n",
      "Gold       2015-12-17\n",
      "Japan      2012-06-01\n",
      "dtype: datetime64[ns]\n",
      "\n",
      " ::  Index on max:\n",
      "America    2017-06-19\n",
      "Emerging   2011-04-26\n",
      "Europe     2014-06-06\n",
      "Gold       2011-08-22\n",
      "Japan      2017-06-02\n",
      "dtype: datetime64[ns]\n",
      "\n",
      " ::  Head:\n",
      "            America  Emerging  Europe    Gold  Japan\n",
      "T                                                   \n",
      "2011-01-03   127.05     48.10   35.47  138.00  44.05\n",
      "2011-01-04   126.98     48.32   35.40  134.75  44.08\n",
      "2011-01-05   127.64     48.20   35.05  134.37  43.76\n",
      " ::  Tail:\n",
      "            America  Emerging  Europe    Gold  Japan\n",
      "T                                                   \n",
      "2017-06-22   242.84     41.28   40.34  118.92  54.09\n",
      "2017-06-23   243.13     41.50   40.39  119.43  54.05\n",
      "2017-06-26   243.29     41.90   40.54  118.36  53.90\n",
      "\n",
      " ::  Correlation matrix:\n",
      "           America  Emerging    Europe      Gold     Japan\n",
      "America   1.000000 -0.453960  0.504256 -0.845116  0.884019\n",
      "Emerging -0.453960  1.000000  0.310990  0.442832 -0.225018\n",
      "Europe    0.504256  0.310990  1.000000 -0.598192  0.676450\n",
      "Gold     -0.845116  0.442832 -0.598192  1.000000 -0.864962\n",
      "Japan     0.884019 -0.225018  0.676450 -0.864962  1.000000\n"
     ]
    }
   ],
   "source": [
    "stats( prices[start:end] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Geometric mean rate\n",
    "\n",
    "David E. Shaw, famous for his proprietary hedge fund, remarked that \n",
    "one of the most important equations in finance is the penalization \n",
    "of arithmetic mean by one-half of variance:\n",
    "\n",
    "$$ g = \\mu - \\frac{\\sigma^2}{2} $$\n",
    "\n",
    "which turns out to be a second-order approximation of geometric mean rate.\n",
    "It is good enough to maximize, before considering \n",
    "mean-variance trade-offs.\n",
    "\n",
    "However, many assets have leptokurtotic returns (\"fat-tails\") and so\n",
    "a more accurate approximation of the geometric mean rate is needed\n",
    "which considers the fourth central moment called *kurtosis* as risk.\n",
    "Details are given on Gaussian mixtures in our research\n",
    "at https://git.io/gmix\n",
    "\n",
    "For maximimizing wealth over many periods, the geometric mean rate\n",
    "as an objective metric should be optimized, rather than the\n",
    "arithmetic mean rate (which precludes risk).\n",
    "\n",
    "A Boltzmann portfolio maximizes the weighted geometric mean rate\n",
    "of its underlying assets. We shall assume that the covariance structure,\n",
    "e.g. the correlations, of the geometric mean rates is similar\n",
    "to the arithmetic mean rates.\n",
    "\n",
    "The source code shown by `groupgemrat??` tells us the output format:\n",
    "the **geometric** mean return, followed by \n",
    "the **arithmetic mean return, volatility, and Pearson kurtosis**, \n",
    "then yearly frequency, sample size, and key -- in list format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[9.1626, 10.2097, 14.7321, 7.6681, 256, 1629, 'America'],\n",
       " [-4.9559, -2.1686, 21.8959, 6.1498, 256, 1629, 'Emerging'],\n",
       " [-1.4118, 2.0996, 24.0893, 9.2461, 256, 1629, 'Europe'],\n",
       " [-4.0893, -2.4126, 17.0602, 8.7778, 256, 1629, 'Gold'],\n",
       " [1.335, 3.1714, 18.5607, 6.7887, 256, 1629, 'Japan']]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Geometric mean rates, non-overlapping, annualized:\n",
    "gems = groupgemrat( prices[start:end], yearly=256, order=False, n=4 )\n",
    "gems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The fourth element in each sublist gives us the kurtosis statistic\n",
    "where 3 is theoretically expected if the distribution is Gaussian.\n",
    "Anything much higher is considered \"leptokurtoic.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Portfolio characteristics\n",
    "\n",
    "At this point, if we were constrained to pick a single agent,\n",
    "we would pick the one with highest geometric mean rate.\n",
    "If short sales were permitted, we would need to\n",
    "evaluate the absolute values of the geometric mean rates.\n",
    "\n",
    "This constrained single-period example makes it clear that\n",
    "we are merely estimating the best allocation of a portfolio,\n",
    "not the timing or size of the trade. Extreme compositions could\n",
    "signal a *bubble* or *anti-bubble*, and a discretionary\n",
    "warning should be issued by the code at the decision stage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['America', 'Emerging', 'Europe', 'Gold', 'Japan']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  By construction, keys will be alphabetically sorted:\n",
    "keys = list(prices.columns)\n",
    "keys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 9.1626],\n",
       "       [-4.9559],\n",
       "       [-1.4118],\n",
       "       [-4.0893],\n",
       "       [ 1.335 ]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Gather just the geometric mean rates into an array:\n",
    "rates = np.array([item[0] for item in gems]).reshape(len(gems), 1)\n",
    "rates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#  Compute the COVARIANCE matrix V:\n",
    "V = covdiflog( prices[start:end] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.    0.82  0.82 -0.03  0.7 ]\n",
      " [ 0.82  1.    0.79  0.12  0.67]\n",
      " [ 0.82  0.79  1.    0.05  0.66]\n",
      " [-0.03  0.12  0.05  1.   -0.  ]\n",
      " [ 0.7   0.67  0.66 -0.    1.  ]]\n"
     ]
    }
   ],
   "source": [
    "#  Values within the covariance matrix itself is hard\n",
    "#  to interpret, so we show the Pearson correlation coefficients,\n",
    "#  rounded to n decimal places:\n",
    "print( cov2cor(V, n=2) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This correlation matrix pertains to the differential\n",
    "between logged prices, *not* the prices themselves\n",
    "(which was summarized earlier).\n",
    "\n",
    "We displayed the correlation matrix as a courtesy because\n",
    "the covariance matrix itself does not have such a\n",
    "readable interpretation of the rates of return.\n",
    "\n",
    "Mathematically and computationally, we shall work henceforth\n",
    "with $V$, the covariance matrix.\n",
    "\n",
    "As expected, the equities group is inter-correlated,\n",
    "whereas Gold rates stand apart."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Weights from the covariance matrix\n",
    "\n",
    "We now turn our attention to the weights associated with the **Global\n",
    "Minimum Variance Portfolio**. Its derivation is well-known,\n",
    "e.g. Cochrane (2005), chp. 5, p.83:\n",
    "\n",
    "$$ \\mathbf{w} = \\frac{V^{-1}\\mathbf{1}} { \\mathbf{1}^\\top V^{-1} \\mathbf{1} } $$\n",
    "\n",
    "Note that the weights are solely dependent on the covariance matrix $V$.\n",
    "There are no constraints involved.\n",
    "\n",
    "The Boltzmann portfolio is *informed* by the Lagrangian formulation\n",
    "of the covariance structure, not in the static single-period sense,\n",
    "but rather in its dynamic evolution over time.\n",
    "Weights will be revised by an online algorithm which\n",
    "tracks covariance, e.g. the Kalman filter, without\n",
    "performing quadratic optimization offline.\n",
    "\n",
    "We shall take only what we truly need in terms of weight coefficients\n",
    "(to be handled by `trimit()` later)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "##  Uncomment to see defined function written in numpy...\n",
    "#  weighcov??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.87034542],\n",
       "       [-0.2267291 ],\n",
       "       [-0.19612603],\n",
       "       [ 0.40540278],\n",
       "       [ 0.14710693]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "globalw = weighcov( V )\n",
    "globalw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The weights at this point are for the\n",
    "*Global Minimum Variance Porfolio* (GMVP).\n",
    "Negative weights imply shorting the underlying asset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deciding on the expert(s)\n",
    "\n",
    "We can now assign scores to our agents which will help to discern the expert(s).\n",
    "A score will be the individual geometric mean rate weighted by the its GMVP weight."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 7.97462698],\n",
       "       [ 1.12364674],\n",
       "       [ 0.27689073],\n",
       "       [-1.65781357],\n",
       "       [ 0.19638775]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Using arrays, we are multiplying element-wise:\n",
    "scores_gmvp = globalw * rates\n",
    "scores_gmvp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.9137386200179298"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Expected GLOBAL portfolio return with UNRESTRICTED short sales:\n",
    "np.sum(scores_gmvp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The weights indicating short positions are usually\n",
    "paired with negative geometric returns.\n",
    "\n",
    "Considering the covariance matrix,\n",
    "an asset which has consistently very poor growth performance\n",
    "may obtain the best score if shorting is permitted."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Trimming weights: Dealing with short sales\n",
    "\n",
    "Negative weights imply the underlying assets should be shorted.\n",
    "A Boltzmann portfolio only considers the weights as ***advisory***\n",
    "in recognition of the fact that the covariance structure is unstable.\n",
    "\n",
    "We may want to limit short sales at -0.30 weight, or perhaps ignore tiny\n",
    "positions for rebalancing purposes in a multiple-period setting.\n",
    "\n",
    "We earlier specified a threshold weight: `MIN_weight`.\n",
    "The `level` argument resets weights under the floor to a specific value,\n",
    "here we pick zero as level.\n",
    "The trimmed weights will then need to be renormalized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m \u001b[0mtrimit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mDocstring:\u001b[0m For an iterable, accept values > floor, else set to level.\n",
       "\u001b[0;31mFile:\u001b[0m      ~/Dropbox/ipy/fecon235/lib/ys_prtf_boltzmann.py\n",
       "\u001b[0;31mType:\u001b[0m      function\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "trimit?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.87034542],\n",
       "       [ 0.        ],\n",
       "       [ 0.        ],\n",
       "       [ 0.40540278],\n",
       "       [ 0.14710693]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights = trimit( globalw, MIN_weight, 0 )\n",
    "weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.61168942],\n",
       "       [ 0.        ],\n",
       "       [ 0.        ],\n",
       "       [ 0.28492203],\n",
       "       [ 0.10338855]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weights = renormalize(weights)\n",
    "weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5.60466546],\n",
       "       [-0.        ],\n",
       "       [-0.        ],\n",
       "       [-1.16513167],\n",
       "       [ 0.13802371]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scores_rentrim = weights * rates\n",
    "scores_rentrim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.5775574975845643"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Expected portfolio return with rentrim weights:\n",
    "np.sum(scores_rentrim)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introducing Boltzmann and softmax\n",
    "\n",
    "Suppose there are $N$ agents, and we want to estimate the probabilities\n",
    "of an agent being the best in the game. So far our computations have not\n",
    "directly estimated probabilities, instead real-valued scores were produced.\n",
    "However, from these scores one can compute the probabilities using the\n",
    "softmax function. We can work with functions which map to the unconstrained space\n",
    "of scores, and then map those scores to the space of probability vectors.\n",
    "One can view this as a generalization of logistic regression.\n",
    "\n",
    "In statistical physics, the softmax function gives the probability of\n",
    "an atom being found in a quantum state of energy when the atom is part of an\n",
    "ensemble that has reached thermal equilibrium.\n",
    "This is known as the **Boltzmann** distribution.\n",
    "\n",
    "In the field of reinforcement learning, the softmax function is used to\n",
    "convert values into action probabilities.\n",
    "A positive parameter $\\tau$ called the temperature is introduced\n",
    "to divide through each value. It is a scaling operation such that\n",
    "high temperatures cause corresponding actions to be equi-probable.\n",
    "Low temperatures cause a greater difference in selection probability\n",
    "for actions that differ in their value estimates.\n",
    "At low temperatures, the probability of the action with the\n",
    "highest expected reward tends to 1.\n",
    "\n",
    "Suppose $\\mathbf{p}$ is our softmax vector, then let its j-th element be:\n",
    "\n",
    "$$ p_j = \\frac {\\exp({s_{j}/\\tau})} {\\sum_{i=1}^{N}\\exp({s_{i}/\\tau})} $$\n",
    "\n",
    "where $s_{j}$ is the score for the j-th asset.\n",
    "[Note: our softmax() code is numerically stabilized for very large scores,\n",
    "and uses a scaled version of $\\tau$ called `temp`.]\n",
    "See References below for more details.\n",
    "Mathematical properties and justification for portfolios\n",
    "will be given in Part 4 of our notebook series.\n",
    "\n",
    "Aside from its theoretical pedigree, **why would softmax be helpful\n",
    "in optimizing the terminal value of our portfolio?**\n",
    "Its elements, interpreted as probabilities, are essentially a function\n",
    "of the *exponential growth* associated with the temperature\n",
    "dampened scores. In other words, the *probabilities will tilt\n",
    "favorably towards assets with the greatest growth potential.*\n",
    "\n",
    "Softmax is also known as the \"***normalized exponential function***.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.7041],\n",
       "       [ 0.0801],\n",
       "       [ 0.0801],\n",
       "       [ 0.051 ],\n",
       "       [ 0.0846]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "problist = softmax( scores_rentrim, temp=TEMPERATURE )[-1]\n",
    "probs = np.array( problist ).reshape(len(problist), 1)\n",
    "probs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to note that probs here used a constant `TEMPERATURE` value.\n",
    "In practice, the temperature can be varied over multiple time periods\n",
    "(to be discussed further in Part 2).\n",
    "\n",
    "*Knowing which agents are likely to be experts, we can now\n",
    "feedback this information to recompute the weights.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.43069052],\n",
       "       [ 0.        ],\n",
       "       [ 0.        ],\n",
       "       [ 0.01453102],\n",
       "       [ 0.00874667]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pweights = probs * weights\n",
    "pweights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Renormalization of pweights is necessary, but also trimit() is helpful,\n",
    "followed by renormalize() again -- to satisfy our own trading restrictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.94872395],\n",
       "       [ 0.        ],\n",
       "       [ 0.        ],\n",
       "       [ 0.0320089 ],\n",
       "       [ 0.01926714]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pweights = renormalize(trimit(renormalize(pweights), MIN_weight, 0))\n",
    "pweights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 8.6927781 ],\n",
       "       [-0.        ],\n",
       "       [-0.        ],\n",
       "       [-0.130894  ],\n",
       "       [ 0.02572164]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scores = pweights * rates\n",
    "scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8.5876057401765546"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Expected geometric mean rate for our Boltzmann portfolio:\n",
    "np.sum(scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SUMMARY\n",
    "\n",
    "Hopefully, our flowchart in the Introduction has become explicitly clear:\n",
    "\n",
    "\n",
    "```\n",
    "    prices ---> cov ---> globalw\n",
    "      |                    |\n",
    "      |                  trimit  <-- floor\n",
    "      |                  renormalize\n",
    "      |                    |\n",
    "      v                    v\n",
    "      |                    |\n",
    "    gemrat              weights\n",
    "      |                    |\n",
    "      |________scores______|\n",
    "                 |\n",
    "                 |                   Boltzmann\n",
    "      temp --> softmax --> probs --> pweights\n",
    "\n",
    "\n",
    "\n",
    "```\n",
    "\n",
    "The algorithmic summary is given by `boltzportfolio()`, and the\n",
    "supporting functions are given in the module `lib/ys_prtf_boltzmann.py`,\n",
    "The condensed versions will be thoroughly employed in Part 2\n",
    "of the notebook series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m \u001b[0mboltzportfolio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myearly\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m256\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m55\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.01\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mSource:\u001b[0m   \n",
       "\u001b[0;32mdef\u001b[0m \u001b[0mboltzportfolio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myearly\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m256\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m55\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.01\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;34m'''MAIN: SUMMARY of Boltzmann portfolio, rounded to n-decimal places.\u001b[0m\n",
       "\u001b[0;34m       Return list where computed values are Python floats, not array type, e.g.\u001b[0m\n",
       "\u001b[0;34m           [2.7833,\u001b[0m\n",
       "\u001b[0;34m            [[0.6423, 2.05, 'America'],\u001b[0m\n",
       "\u001b[0;34m             [0.0, -11.17, 'Emerging'],\u001b[0m\n",
       "\u001b[0;34m             [0.0, -10.47, 'Europe'],\u001b[0m\n",
       "\u001b[0;34m             [0.3577, 4.1, 'Gold'],\u001b[0m\n",
       "\u001b[0;34m             [0.0, -4.99, 'Japan']]]\u001b[0m\n",
       "\u001b[0;34m       The portfolio's geometric mean rate is included first.\u001b[0m\n",
       "\u001b[0;34m       Each sub-sublist will consist of weight, rate, and key.\u001b[0m\n",
       "\u001b[0;34m       The order of keys from the dataframe is preserved.\u001b[0m\n",
       "\u001b[0;34m    '''\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mrates\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgemratarr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myearly\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mglobalw\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mweighcovdata\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mweights\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrentrim\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mglobalw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mpweights\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mweighsoft\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mweights\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrates\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#      ---- so far should be the same as boltzweigh()\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpweights\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mrates\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mgrat\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mround\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mscores\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mkeys\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  wrk, i.e. \"weight, rate, key\", is a list of lists:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mwrk\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mtools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mroundit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrates\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkeys\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mecho\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mw\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpweights\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m \u001b[0mgrat\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwrk\u001b[0m \u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mFile:\u001b[0m      ~/Dropbox/ipy/fecon235/lib/ys_prtf_boltzmann.py\n",
       "\u001b[0;31mType:\u001b[0m      function\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "boltzportfolio??"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***The numerical computations of this entire notebook,\n",
    "including the inversion of the covariance matrix\n",
    "and the softmax evaluation, can be replicated as follows:***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[8.5876,\n",
       " [[0.9487, 9.1626, 'America'],\n",
       "  [0.0, -4.9559, 'Emerging'],\n",
       "  [0.0, -1.4118, 'Europe'],\n",
       "  [0.032, -4.0893, 'Gold'],\n",
       "  [0.0193, 1.335, 'Japan']]]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  See the docstring to decipher the format:\n",
    "boltzportfolio(prices[start:end], yearly=256, temp=TEMPERATURE, floor=MIN_weight, level=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Noteworthy: Gold which had a *negative* geometric mean rate for itself\n",
    "receives a *positive* pweight greater than Japan,\n",
    "due to the covariance structure and our trimit specification. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## References\n",
    "\n",
    "- John H. Cochrane, 2005 revised ed., *Asset Pricing*, Princeton U. Press.\n",
    "\n",
    "- On softmax:\n",
    "    - https://en.wikipedia.org/wiki/Softmax_function\n",
    "    - https://compute.quora.com/What-is-softmax\n",
    "    - http://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative\n",
    "    - http://cs231n.github.io/linear-classify/#softmax\n",
    "    - https://en.wikipedia.org/wiki/Reinforcement_learning"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}