nb/prtf-boltzmann-2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boltzmann portfolios\n",
    "\n",
    "### Part 2: Parametization and dynamics of weights\n",
    "\n",
    "We develop an alternative to the traditional mean-variance\n",
    "framework (\"*Markowitz*\" portfolios) called ***Boltzmann*** portfolios\n",
    "which handle uncertainty from the standpoint of cross-entropy\n",
    "and optimal sequential decisions.\n",
    "The improved result is a faster online algorithm which is more robust.\n",
    "\n",
    "Markowitz portfolios are optimal in the arithmetic mean-variance\n",
    "framework for a *single-period*. They are fragile to changing\n",
    "market conditions, much like elegant battle strategies which\n",
    "crumble under harsh war conditions.\n",
    "In contrast, Boltzmann portfolios are designed to be\n",
    "adaptive over multiple periods to maximize final wealth.\n",
    "Techniques have been borrowed from Bayesian and\n",
    "reinforcement learning.\n",
    "\n",
    "Here in Part 2, we explore variations in parametization and weights over time.\n",
    "Sequential decisions based on such information will be covered in Part 3\n",
    "(where data windowing will be replaced by online filtration).\n",
    "Part 4 will cover the mathematical proof of concept, but here\n",
    "are many numerical lessons which will help guide us.\n",
    "\n",
    "Appendix 1 herein examines in detail how the `boltzportfolio()`\n",
    "algorithm robustly handled leptokurtotic events\n",
    "from the global credit crisis of August 2011."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Dependencies:*\n",
    "\n",
    "- Repository: https://github.com/rsvp/fecon235\n",
    "     \n",
    "*CHANGE LOG*\n",
    "\n",
    "    2017-07-21  Edit per stable Part 1. Add Appendix 1.\n",
    "    2017-06-29  Investigate temperature parametization, weight dynamics.\n",
    "    2017-06-28  Functions moved to ys_prtf_boltzmann module.\n",
    "    2017-06-27  First version, functions numerically tested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from fecon235.fecon235 import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ::  Python 2.7.13\n",
      " ::  IPython 5.1.0\n",
      " ::  jupyter_core 4.2.1\n",
      " ::  notebook 4.1.0\n",
      " ::  matplotlib 1.5.1\n",
      " ::  numpy 1.11.0\n",
      " ::  scipy 0.17.0\n",
      " ::  sympy 1.0\n",
      " ::  pandas 0.19.2\n",
      " ::  pandas_datareader 0.2.1\n",
      " ::  Repository: fecon235 v5.17.0603 devPrtf\n",
      " ::  Timestamp: 2017-07-21T21:57:22Z\n",
      " ::  $pwd: /media/yaya/virt15h/virt/dbx/Dropbox/ipy/fecon235/nb\n"
     ]
    }
   ],
   "source": [
    "#  PREAMBLE-p6.15.1223d :: Settings and system details\n",
    "from __future__ import absolute_import, print_function, division\n",
    "system.specs()\n",
    "pwd = system.getpwd()   # present working directory as variable.\n",
    "print(\" ::  $pwd:\", pwd)\n",
    "#  If a module is modified, automatically reload it:\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "#       Use 0 to disable this feature.\n",
    "\n",
    "#  Notebook DISPLAY options:\n",
    "#      Represent pandas DataFrames as text; not HTML representation:\n",
    "import pandas as pd\n",
    "pd.set_option( 'display.notebook_repr_html', False )\n",
    "from IPython.display import HTML # useful for snippets\n",
    "#  e.g. HTML('<iframe src=http://en.mobile.wikipedia.org/?useformat=mobile width=700 height=350></iframe>')\n",
    "from IPython.display import Image \n",
    "#  e.g. Image(filename='holt-winters-equations.png', embed=True) # url= also works\n",
    "from IPython.display import YouTubeVideo\n",
    "#  e.g. YouTubeVideo('1j_HxD4iLn8', start='43', width=600, height=400)\n",
    "from IPython.core import page\n",
    "get_ipython().set_hook('show_in_pager', page.as_hook(page.display_page), 0)\n",
    "#  Or equivalently in config file: \"InteractiveShell.display_page = True\", \n",
    "#  which will display results in secondary notebook pager frame in a cell.\n",
    "\n",
    "#  Generate PLOTS inside notebook, \"inline\" generates static png:\n",
    "%matplotlib inline   \n",
    "#          \"notebook\" argument allows interactive zoom and resize."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "***It is important to first understand Part 1 of this series,\n",
    "and the narrative behind the code.\n",
    "We dive in with its condensed content, after constructing a dataframe.***\n",
    "\n",
    "Full details are in the module, `lib/ys_prtf_boltzmann.py`.\n",
    "Information on any function **foo()** can be displayed in a cell\n",
    "by `foo?` or with more details by `foo??`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download data and construct a dataframe\n",
    "\n",
    "We retrieve the following data of daily frequency\n",
    "representing equities worldwide and gold by five ETF securities: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'America': 's4spy',\n",
       " 'Emerging': 's4eem',\n",
       " 'Europe': 's4ezu',\n",
       " 'Gold': 's4gld',\n",
       " 'Japan': 's4ewj'}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Convenient dictionary set in fecon235.py,\n",
    "#  where key is world region, and value is its fecon235 data code:\n",
    "world4d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#  Or manually specify your own dictionary here:\n",
    "prices_dic = world4d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ::  Retrieved from Google Finance: SPY\n",
      " ::  Retrieved from Google Finance: EEM\n",
      " ::  Retrieved from Google Finance: EZU\n",
      " ::  Retrieved from Google Finance: GLD\n",
      " ::  Retrieved from Google Finance: EWJ\n"
     ]
    }
   ],
   "source": [
    "#  Download data into a dataframe, alphabetically by key:\n",
    "prices = groupget( prices_dic, maxi=3650 )\n",
    "#  ... about ten years worth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#  Define start and end for prices, when applicable:\n",
    "start = '2011-01-01'\n",
    "end = '2017-06-26'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visual summary of Part 1\n",
    "\n",
    "\n",
    "```\n",
    "    prices ---> cov ---> globalw\n",
    "      |                    |\n",
    "      |                  trimit  <-- floor\n",
    "      |                  renormalize\n",
    "      |                    |\n",
    "      v                    v\n",
    "      |                    |\n",
    "    gemrat              weights\n",
    "      |                    |\n",
    "      |________scores______|\n",
    "                 |\n",
    "                 |                   Boltzmann\n",
    "      temp --> softmax --> probs --> pweights\n",
    "\n",
    "             \n",
    "```\n",
    "\n",
    "where the function **boltzportfolio()** unifies the supporting functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m \u001b[0mboltzportfolio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myearly\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m256\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m55\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.01\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mSource:\u001b[0m   \n",
       "\u001b[0;32mdef\u001b[0m \u001b[0mboltzportfolio\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myearly\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m256\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m55\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0.01\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;34m'''MAIN: SUMMARY of Boltzmann portfolio, rounded to n-decimal places.\u001b[0m\n",
       "\u001b[0;34m       Return list where computed values are Python floats, not array type, e.g.\u001b[0m\n",
       "\u001b[0;34m           [2.7833,\u001b[0m\n",
       "\u001b[0;34m            [[0.6423, 2.05, 'America'],\u001b[0m\n",
       "\u001b[0;34m             [0.0, -11.17, 'Emerging'],\u001b[0m\n",
       "\u001b[0;34m             [0.0, -10.47, 'Europe'],\u001b[0m\n",
       "\u001b[0;34m             [0.3577, 4.1, 'Gold'],\u001b[0m\n",
       "\u001b[0;34m             [0.0, -4.99, 'Japan']]]\u001b[0m\n",
       "\u001b[0;34m       The portfolio's geometric mean rate is included first.\u001b[0m\n",
       "\u001b[0;34m       Each sub-sublist will consist of weight, rate, and key.\u001b[0m\n",
       "\u001b[0;34m       The order of keys from the dataframe is preserved.\u001b[0m\n",
       "\u001b[0;34m    '''\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mrates\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgemratarr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0myearly\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mglobalw\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mweighcovdata\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mweights\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrentrim\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mglobalw\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mpweights\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mweighsoft\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mweights\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrates\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#      ---- so far should be the same as boltzweigh()\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpweights\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mrates\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mgrat\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mround\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mscores\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mkeys\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataframe\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  wrk, i.e. \"weight, rate, key\", is a list of lists:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mwrk\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mtools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mroundit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrates\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkeys\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mecho\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mw\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpweights\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m \u001b[0mgrat\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mwrk\u001b[0m \u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mFile:\u001b[0m      ~/Dropbox/ipy/fecon235/lib/ys_prtf_boltzmann.py\n",
       "\u001b[0;31mType:\u001b[0m      function\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "boltzportfolio??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[8.5876,\n",
       " [[0.9487, 9.1626, 'America'],\n",
       "  [0.0, -4.9559, 'Emerging'],\n",
       "  [0.0, -1.4118, 'Europe'],\n",
       "  [0.032, -4.0893, 'Gold'],\n",
       "  [0.0193, 1.335, 'Japan']]]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Part 1 summarized all its computations as follows:\n",
    "prtf = boltzportfolio( prices[start:end], temp=55, floor=0.01, level=0, n=4 )\n",
    "prtf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first element is the *estimated* geometric mean rate of the entire portfolio.\n",
    "Respective weights of the underlying assets\n",
    "are mentioned in the sub-sublist along with the individual geometric mean rates."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Softmax and temperature\n",
    "\n",
    "In **reinforcement learning**, the softmax function is used to\n",
    "convert values into action probabilities.\n",
    "A positive parameter called the temperature is introduced\n",
    "to divide through each value. It is a scaling operation such that\n",
    "high temperatures cause corresponding actions to be equi-probable.\n",
    "Low temperatures cause a greater difference in selection probability\n",
    "for actions that differ in their value estimates.\n",
    "At low temperatures, the probability of the action with the\n",
    "highest expected reward tends to 1.\n",
    "The mathematical theory involves cross-entropy which will be\n",
    "discussed in Part 4.\n",
    "\n",
    "The **softmax()** function is the essential part of `weighsoft()`,\n",
    "which is invoked in `boltzportfolio()`,\n",
    "and many algorithms in machine learning (see References below)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m \u001b[0mweighsoft\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mweights\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrates\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mSource:\u001b[0m   \n",
       "\u001b[0;32mdef\u001b[0m \u001b[0mweighsoft\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0mweights\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrates\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;34m'''Given weights, compute pweights as array by softmax transform.'''\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mscores\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mweights\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mrates\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mproblist\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmlearn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msoftmax\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0mscores\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mprobs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0mproblist\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mproblist\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  Revise weights based on softmax probabilities:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mpweights\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mprobs\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mweights\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  Then appropriately adjust:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;32mreturn\u001b[0m \u001b[0mrentrim\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mrenormalize\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpweights\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mFile:\u001b[0m      ~/Dropbox/ipy/fecon235/lib/ys_prtf_boltzmann.py\n",
       "\u001b[0;31mType:\u001b[0m      function\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "weighsoft??"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m \u001b[0msoftmax\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m55\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mSource:\u001b[0m   \n",
       "\u001b[0;32mdef\u001b[0m \u001b[0msoftmax\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0mit\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m55\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;34m'''Softmax probabilities for iterable where temp sets temperature tau.\u001b[0m\n",
       "\u001b[0;34m       Temperature tau is set as a temp percent of ensemble mean so that\u001b[0m\n",
       "\u001b[0;34m       the scaling of tau works well across many different scenarios.\u001b[0m\n",
       "\u001b[0;34m       Experiment with temp around 40 to 70; higher temp (100+)\u001b[0m\n",
       "\u001b[0;34m       will make it-scores more equi-probable, whereas probabilities\u001b[0m\n",
       "\u001b[0;34m       can be sharpened by decreasing temp towards 1.\u001b[0m\n",
       "\u001b[0;34m       Setting temp to 0 results in generic softmax without temperature.\u001b[0m\n",
       "\u001b[0;34m       Results are rounded to n decimal places.\u001b[0m\n",
       "\u001b[0;34m    >>> softmax( [ 16, 8, 4, 0, -8, -16 ], temp=200, n=4 )\u001b[0m\n",
       "\u001b[0;34m    [0, 16, 0.2598, 200, [0.2598, 0.2001, 0.1757, 0.1542, 0.1188, 0.0915]]\u001b[0m\n",
       "\u001b[0;34m    >>> softmax( [ 16, 8, 4, 0, -8, -16 ], temp=50, n=4 )\u001b[0m\n",
       "\u001b[0;34m    [0, 16, 0.5733, 50, [0.5733, 0.2019, 0.1198, 0.0711, 0.0251, 0.0088]]\u001b[0m\n",
       "\u001b[0;34m    >>> softmax( [ 16, 8, 4, 0, -8, -16 ], temp=30, n=4 )\u001b[0m\n",
       "\u001b[0;34m    [0, 16, 0.7773, 30, [0.7773, 0.1365, 0.0572, 0.024, 0.0042, 0.0007]]\u001b[0m\n",
       "\u001b[0;34m    >>> softmax( [ 16, 8, 4, 0, -8, -16 ], temp=1, n=4 )\u001b[0m\n",
       "\u001b[0;34m    [0, 16, 1.0, 1, [1.0, 0.0, 0.0, 0.0, 0.0, 0.0]]\u001b[0m\n",
       "\u001b[0;34m    >>> softmax( [ 16, 8, 4, 0, -8, -16 ], temp=0, n=4 )\u001b[0m\n",
       "\u001b[0;34m    [0, 16, 0.9997, 0, [0.9997, 0.0003, 0.0, 0.0, 0.0, 0.0]]\u001b[0m\n",
       "\u001b[0;34m    >>> softmax( [ 16, 16 ], temp=200, n=4 )\u001b[0m\n",
       "\u001b[0;34m    [0, 16, 0.5, 200, [0.5, 0.5]]\u001b[0m\n",
       "\u001b[0;34m    >>> softmax( [ 16, 15, -8 ], temp=50, n=4 )\u001b[0m\n",
       "\u001b[0;34m    [0, 16, 0.5587, 50, [0.5587, 0.4395, 0.0018]]\u001b[0m\n",
       "\u001b[0;34m    '''\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  Reference: https://compute.quora.com/What-is-softmax\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  Convert iterable to numpy array, then find index of its maximum value:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0marr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtoar\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0mit\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0midmax\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margmax\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0marr\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mhardmax\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marr\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0midmax\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  Without loss of generality for mathematically defined softmax,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  subtracting an arbitrary constant from each it-element\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  always helps NUMERICAL STABILITY, so let it be hardmax:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0marrstable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0marr\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mhardmax\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0;31m#  Important to note arrstable will consist of maximum(s) represented\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0;31m#  by zero, and all other values will be necessarily negative.\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;32mif\u001b[0m \u001b[0mtemp\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0mavg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmean\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marrstable\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0;32mif\u001b[0m \u001b[0mavg\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0mtau\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mabs\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0mavg\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mtemp\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0;36m100.0\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0;31m#  Let temperature be POSITIVE and temp percent of ensemble mean.\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0;31m#  Edge case: avg will be zero if it-scores are all equal,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0;31m#  which implies they are equi-probable, so any tau should do,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0;31m#  but tau must be NON-ZERO to avoid division error next.\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m            \u001b[0mtau\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1.0\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0;31m#  Whenever temp is set to 0, False, or None => GENERIC softmax.\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0;31m#  Also negative temp will be treated as generic softmax.\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0mtemp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m   \u001b[0;31m# Prefer the numerical eqivalent for return below.\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m        \u001b[0mtau\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1.0\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#  MATHEMATICALLY, (Boltzmann) softmax is defined as follows:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mexpit\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexp\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0marrstable\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mtau\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0msum_expit\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msum\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0mexpit\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0msoftmax_exact\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mexpit\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0msum_expit\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;31m#                      roundit will output a list, not an array:\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0msoftmax_approx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtools\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mroundit\u001b[0m\u001b[0;34m(\u001b[0m \u001b[0msoftmax_exact\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mecho\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mFalse\u001b[0m \u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mhardprob\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msoftmax_approx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0midmax\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0;32mreturn\u001b[0m \u001b[0;34m[\u001b[0m \u001b[0midmax\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhardmax\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhardprob\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtemp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msoftmax_approx\u001b[0m \u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mFile:\u001b[0m      ~/Dropbox/ipy/fecon235/lib/ys_mlearn.py\n",
       "\u001b[0;31mType:\u001b[0m      function\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "softmax??"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Temperature variations\n",
    "\n",
    "We vary the scaled temperature `temp`\n",
    "(proxy for our uncertainty over the estimators)\n",
    "to see its effect on the Boltzmann portfolio weights,\n",
    "`pweight` for each agent, and the\n",
    "geometric mean rate for the portfolio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "            America  Emerging  Europe  Gold  Japan\n",
       "T                                                 \n",
       "2007-07-25   151.61     46.59   58.96  66.9  58.64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Show first date of the dataframe:\n",
    "head(prices, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vary T where shorts > -10\n",
    "\n",
    "Note: ***Short positions*** in this notebook are for mathematical illustration.\n",
    "They ignore limitations imposed by regulations\n",
    "and the costs introduced by borrowing.\n",
    "In practice, one would not use the listed securities,\n",
    "but rather derivatives based on regional indexes to gain *leverage*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 \t 29.84 \t [0.0, -1.0, -0.0, 0.0, 0.0]\n",
      "21 \t 33.18 \t [0.0, -0.89, -0.17, 0.06, 0.0]\n",
      "41 \t 42.86 \t [0.01, -0.94, -0.34, 0.26, 0.01]\n",
      "61 \t 55.37 \t [0.05, -1.11, -0.49, 0.51, 0.03]\n",
      "81 \t 73.37 \t [0.18, -1.41, -0.69, 0.84, 0.08]\n",
      "101 \t 103.81 \t [0.46, -1.97, -1.03, 1.37, 0.17]\n",
      "121 \t 167.68 \t [1.1, -3.18, -1.73, 2.44, 0.37]\n",
      "141 \t 392.2 \t [3.43, -7.47, -4.2, 6.18, 1.06]\n",
      "161 \t 22.84 \t [0.58, -0.32, -0.32, 0.89, 0.17]\n",
      "181 \t 283.42 \t [3.69, -5.51, -3.22, 5.03, 1.01]\n",
      "201 \t 159.95 \t [2.4, -3.14, -1.87, 2.97, 0.64]\n",
      "221 \t 113.85 \t [1.93, -2.26, -1.36, 2.2, 0.49]\n",
      "241 \t 89.63 \t [1.68, -1.8, -1.09, 1.79, 0.42]\n",
      "261 \t 74.96 \t [1.53, -1.52, -0.93, 1.55, 0.37]\n",
      "281 \t 64.91 \t [1.43, -1.33, -0.82, 1.38, 0.34]\n",
      "301 \t 57.74 \t [1.36, -1.2, -0.74, 1.26, 0.32]\n"
     ]
    }
   ],
   "source": [
    "#  Shorts > -10 for 2008-2011\n",
    "\n",
    "for T in range(1, 302, 20):\n",
    "    prtf = boltzportfolio(prices['2008':'2011'], temp=T, floor=-10, level=-10, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( T, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If our estimators were exact and held into the future,\n",
    "at temp=1, the portfolio would be short Emerging markets without leverage.\n",
    "As the temperature increases, we can see wider diffusion\n",
    "over the portfolio weights.\n",
    "\n",
    "The 2008-2011 period was very volatile due to the Great Recession.\n",
    "At temp=141, the portfolio would be highly leveraged\n",
    "shorting Emerging markets while also leveraged on the long side with Gold.\n",
    "\n",
    "***The geometric mean rate of the portfolio is shown by example\n",
    "not to be a convex function of temperature.***\n",
    "This means if we want to optimize the portfolio rate,\n",
    "convex methods can be ruled out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 \t 11.12 \t [1.0, -0.0, -0.0, 0.0, 0.0]\n",
      "21 \t 11.16 \t [1.0, -0.0, -0.0, 0.0, 0.0]\n",
      "41 \t 11.51 \t [1.02, -0.02, -0.02, 0.01, 0.01]\n",
      "61 \t 11.87 \t [1.04, -0.04, -0.05, 0.03, 0.02]\n",
      "81 \t 12.08 \t [1.05, -0.06, -0.08, 0.06, 0.03]\n",
      "101 \t 12.2 \t [1.06, -0.08, -0.1, 0.08, 0.05]\n",
      "121 \t 12.25 \t [1.06, -0.1, -0.13, 0.11, 0.06]\n",
      "141 \t 12.26 \t [1.06, -0.11, -0.14, 0.13, 0.07]\n",
      "161 \t 12.25 \t [1.06, -0.12, -0.16, 0.15, 0.07]\n",
      "181 \t 12.24 \t [1.06, -0.13, -0.17, 0.16, 0.08]\n",
      "201 \t 12.21 \t [1.06, -0.14, -0.18, 0.18, 0.09]\n",
      "221 \t 12.18 \t [1.06, -0.15, -0.19, 0.19, 0.09]\n",
      "241 \t 12.16 \t [1.06, -0.15, -0.2, 0.2, 0.1]\n",
      "261 \t 12.13 \t [1.05, -0.16, -0.21, 0.21, 0.1]\n",
      "281 \t 12.1 \t [1.05, -0.16, -0.21, 0.22, 0.1]\n",
      "301 \t 12.07 \t [1.05, -0.17, -0.22, 0.23, 0.11]\n"
     ]
    }
   ],
   "source": [
    "#  Shorts > -10 for 2011-2014\n",
    "\n",
    "for T in range(1, 302, 20):\n",
    "    prtf = boltzportfolio(prices['2011':'2014'], temp=T, floor=-10, level=-10, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( T, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 2011-2014 period shows that the portfolio rate could perhaps\n",
    "be a concave function of temperature -- which is false as shown\n",
    "in the previous period.\n",
    "In this period, the portfolio is basically long America\n",
    "at all temperatures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 \t 7.81 \t [1.0, -0.0, -0.0, 0.0, 0.0]\n",
      "21 \t 7.8 \t [1.0, -0.0, -0.0, 0.0, 0.0]\n",
      "41 \t 7.62 \t [0.98, -0.01, -0.0, 0.03, 0.01]\n",
      "61 \t 7.28 \t [0.93, -0.03, -0.0, 0.08, 0.02]\n",
      "81 \t 6.94 \t [0.89, -0.05, -0.01, 0.12, 0.04]\n",
      "101 \t 6.66 \t [0.86, -0.06, -0.01, 0.17, 0.05]\n",
      "121 \t 6.42 \t [0.83, -0.07, -0.01, 0.2, 0.05]\n",
      "141 \t 6.23 \t [0.81, -0.08, -0.01, 0.23, 0.06]\n",
      "161 \t 6.08 \t [0.79, -0.09, -0.01, 0.25, 0.07]\n",
      "181 \t 5.95 \t [0.77, -0.1, -0.01, 0.27, 0.07]\n",
      "201 \t 5.84 \t [0.76, -0.1, -0.01, 0.28, 0.07]\n",
      "221 \t 5.75 \t [0.75, -0.11, -0.01, 0.29, 0.08]\n",
      "241 \t 5.67 \t [0.74, -0.11, -0.01, 0.31, 0.08]\n",
      "261 \t 5.61 \t [0.73, -0.11, -0.01, 0.32, 0.08]\n",
      "281 \t 5.55 \t [0.72, -0.12, -0.01, 0.32, 0.08]\n",
      "301 \t 5.49 \t [0.72, -0.12, -0.01, 0.33, 0.08]\n"
     ]
    }
   ],
   "source": [
    "#  Shorts > -10 for 2014-2017\n",
    "\n",
    "for T in range(1, 302, 20):\n",
    "    prtf = boltzportfolio(prices['2014':'2017'], temp=T, floor=-10, level=-10, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( T, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For 2014-2017, the single best asset is America, given temp=1.\n",
    "More diversification invites Gold into the portfolio.\n",
    "Variations due to temperature are bland during this period."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Vary T, for long-only portfolios\n",
    "\n",
    "We will trim positions less than +0.01 in weight."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 \t 12.03 \t [0.0, 0.0, 0.0, 1.0, 0.0]\n",
      "21 \t 12.03 \t [0.0, 0.0, 0.0, 1.0, 0.0]\n",
      "41 \t 11.29 \t [0.01, 0.0, 0.0, 0.97, 0.02]\n",
      "61 \t 9.55 \t [0.06, 0.0, 0.0, 0.9, 0.04]\n",
      "81 \t 7.53 \t [0.12, 0.0, 0.0, 0.82, 0.06]\n",
      "101 \t 5.73 \t [0.18, 0.0, 0.0, 0.74, 0.08]\n",
      "121 \t 4.28 \t [0.23, 0.0, 0.0, 0.68, 0.09]\n",
      "141 \t 3.13 \t [0.28, 0.0, 0.0, 0.63, 0.09]\n",
      "161 \t 2.22 \t [0.31, 0.0, 0.0, 0.59, 0.1]\n",
      "181 \t 1.49 \t [0.34, 0.0, 0.0, 0.56, 0.1]\n",
      "201 \t 0.9 \t [0.37, 0.0, 0.0, 0.53, 0.1]\n",
      "221 \t 0.41 \t [0.38, 0.0, 0.0, 0.51, 0.1]\n",
      "241 \t 0.0 \t [0.4, 0.0, 0.0, 0.49, 0.11]\n",
      "261 \t -0.34 \t [0.42, 0.0, 0.0, 0.48, 0.11]\n",
      "281 \t -0.63 \t [0.43, 0.0, 0.0, 0.46, 0.11]\n",
      "301 \t -0.89 \t [0.44, 0.0, 0.0, 0.45, 0.11]\n"
     ]
    }
   ],
   "source": [
    "#  Long-only for 2008-2011\n",
    "\n",
    "for T in range(1, 302, 20):\n",
    "    prtf = boltzportfolio(prices['2008':'2011'], temp=T, floor=0.01, level=0, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( T, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For long-only portfolio, the 2008-2011 period favored Gold over other assets.\n",
    "\n",
    "***Temperature which is too high, i.e. unnecessary diffusion,\n",
    "can bring a portfolio's rate of return into negative territory.***\n",
    "\n",
    "*Meta-question: what if the geometric mean rate of a portfolio\n",
    "is* **less than the \"risk-free\" rate of government bonds?**\n",
    "Maybe we should withdrawn entirely from that portfolio,\n",
    "or perhaps it is indicating more wealth should be devoted to the portfolio\n",
    "since the components are becoming cheaply priced.\n",
    "This traditionally is considered an **asset allocation** problem\n",
    "where there are different classes of assets such as equities of varying categories,\n",
    "commodities, real estate, bonds of varying credit risks, and plain cash.\n",
    "\n",
    "In Part 3, we will let machine learning techniques help us\n",
    "to answer such meta-questions in the context of learning to make\n",
    "optimal *sequential* rebalancing decisions when switching costs are included.\n",
    "The Boltzmann softmax is still at the core of whatever we call\n",
    "portfolio or asset allocation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 \t 11.12 \t [1.0, 0.0, 0.0, 0.0, 0.0]\n",
      "21 \t 11.12 \t [1.0, 0.0, 0.0, 0.0, 0.0]\n",
      "41 \t 10.93 \t [0.99, 0.0, 0.0, 0.01, 0.0]\n",
      "61 \t 10.28 \t [0.95, 0.0, 0.0, 0.03, 0.02]\n",
      "81 \t 9.71 \t [0.91, 0.0, 0.0, 0.06, 0.03]\n",
      "101 \t 9.2 \t [0.88, 0.0, 0.0, 0.08, 0.04]\n",
      "121 \t 8.79 \t [0.86, 0.0, 0.0, 0.09, 0.05]\n",
      "141 \t 8.45 \t [0.84, 0.0, 0.0, 0.11, 0.06]\n",
      "161 \t 8.16 \t [0.82, 0.0, 0.0, 0.12, 0.06]\n",
      "181 \t 7.92 \t [0.8, 0.0, 0.0, 0.13, 0.06]\n",
      "201 \t 7.72 \t [0.79, 0.0, 0.0, 0.14, 0.07]\n",
      "221 \t 7.55 \t [0.78, 0.0, 0.0, 0.15, 0.07]\n",
      "241 \t 7.4 \t [0.77, 0.0, 0.0, 0.15, 0.07]\n",
      "261 \t 7.27 \t [0.77, 0.0, 0.0, 0.16, 0.07]\n",
      "281 \t 7.16 \t [0.76, 0.0, 0.0, 0.16, 0.08]\n",
      "301 \t 7.06 \t [0.75, 0.0, 0.0, 0.17, 0.08]\n"
     ]
    }
   ],
   "source": [
    "#  Long-only for 2011-2014\n",
    "\n",
    "for T in range(1, 302, 20):\n",
    "    prtf = boltzportfolio(prices['2011':'2014'], temp=T, floor=0.01, level=0, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( T, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 \t 7.81 \t [1.0, 0.0, 0.0, 0.0, 0.0]\n",
      "21 \t 7.81 \t [1.0, 0.0, 0.0, 0.0, 0.0]\n",
      "41 \t 7.58 \t [0.97, 0.0, 0.0, 0.03, 0.0]\n",
      "61 \t 7.05 \t [0.9, 0.0, 0.0, 0.07, 0.02]\n",
      "81 \t 6.61 \t [0.85, 0.0, 0.0, 0.12, 0.03]\n",
      "101 \t 6.24 \t [0.8, 0.0, 0.0, 0.15, 0.04]\n",
      "121 \t 5.96 \t [0.77, 0.0, 0.0, 0.18, 0.05]\n",
      "141 \t 5.73 \t [0.74, 0.0, 0.0, 0.21, 0.06]\n",
      "161 \t 5.55 \t [0.72, 0.0, 0.0, 0.22, 0.06]\n",
      "181 \t 5.4 \t [0.7, 0.0, 0.0, 0.24, 0.06]\n",
      "201 \t 5.27 \t [0.68, 0.0, 0.0, 0.25, 0.07]\n",
      "221 \t 5.17 \t [0.67, 0.0, 0.0, 0.26, 0.07]\n",
      "241 \t 5.08 \t [0.66, 0.0, 0.0, 0.27, 0.07]\n",
      "261 \t 5.01 \t [0.65, 0.0, 0.0, 0.28, 0.07]\n",
      "281 \t 4.94 \t [0.64, 0.0, 0.0, 0.29, 0.07]\n",
      "301 \t 4.88 \t [0.63, 0.0, 0.0, 0.29, 0.07]\n"
     ]
    }
   ],
   "source": [
    "#  Long-only for 2014-2017\n",
    "\n",
    "for T in range(1, 302, 20):\n",
    "    prtf = boltzportfolio(prices['2014':'2017'], temp=T, floor=0.01, level=0, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( T, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "When short positions are restricted, assets with\n",
    "negative geometric mean rates will generally be excluded\n",
    "from a portfolio.\n",
    "However, assets with somewhat negative rates\n",
    "may get included into the portfolio as the\n",
    "temperature increases.\n",
    "\n",
    "For the long-only portfolios above, we notice that\n",
    "the geometric mean rate decreases *monotonically*\n",
    "as a function of temperature.\n",
    "This means we should know\n",
    "how much return we are willing to sacrifice\n",
    "for possible diversification benefits."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Is there an ideal temperature?\n",
    "\n",
    "It may seem subjective, for example,\n",
    "from the samples so far, that temp=55 is picked as default parameter.\n",
    "What are the out-of-sample results?\n",
    "\n",
    "There is no reason why temperature should be fixed over multiple time periods.\n",
    "In sequential decision situations it may helpful to make the temperature *adaptive*\n",
    "to maximize terminal wealth."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evolution of weights over time at fixed temp\n",
    "\n",
    "Here we fix `temp` at 55, and see how portfolio weights\n",
    "evolve over *overlapping* periods.\n",
    "There will be 100 trading days per period,\n",
    "and we shall advance 50 days at each step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100 \t 30.77 \t [0.18, -0.08, -0.06, 0.88, 0.08]\n",
      "150 \t 88.05 \t [0.07, -0.34, -0.07, 1.27, 0.07]\n",
      "200 \t 53.75 \t [0.04, -0.98, 0.01, 1.79, 0.14]\n",
      "250 \t 41.15 \t [0.07, -1.33, 0.04, 0.17, 0.06]\n",
      "300 \t 119.14 \t [0.04, -0.85, -0.43, 0.19, 0.06]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/yaya/Dropbox/ipy/fecon235/lib/ys_gauss_mix.py:225: RuntimeWarning: invalid value encountered in log\n",
      "  terms = [ np.log(mu_return), 0, 0, 0, 0 ]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "350 \t 186.07 \t [0.02, -0.46, -0.65, 0.06, 0.02]\n",
      "400 \t 235.76 \t [0.02, -1.03, -0.39, 0.37, 0.02]\n",
      "450 \t 18.78 \t [0.33, -0.01, -0.06, 0.71, 0.03]\n",
      "500 \t 41.52 \t [0.88, -0.01, -0.02, 0.11, 0.04]\n",
      "550 \t 34.52 \t [0.81, -0.01, -0.01, 0.17, 0.04]\n",
      "600 \t 36.36 \t [0.93, -0.02, -0.02, 0.07, 0.03]\n",
      "650 \t 155.06 \t [3.74, -0.31, -2.83, 0.3, 0.1]\n",
      "700 \t 70.39 \t [1.89, -0.12, -0.84, 0.03, 0.04]\n",
      "750 \t 161.05 \t [0.19, -1.2, -2.36, 4.25, 0.12]\n",
      "800 \t 10.71 \t [0.14, -0.02, -0.02, 0.77, 0.13]\n",
      "850 \t 34.8 \t [0.92, -0.01, -0.02, 0.1, 0.01]\n",
      "900 \t 40.9 \t [1.02, -0.03, -0.01, 0.02, -0.0]\n",
      "950 \t 25.61 \t [0.93, -0.01, -0.01, 0.1, -0.0]\n",
      "1000 \t 40.48 \t [0.1, -0.04, -0.19, 1.12, 0.01]\n",
      "1050 \t 149.63 \t [0.02, -0.96, -0.37, 0.27, 0.05]\n",
      "1100 \t 102.07 \t [0.06, -0.46, -0.78, 0.15, 0.03]\n",
      "1150 \t 41.26 \t [0.93, -0.02, -0.03, 0.05, 0.07]\n",
      "1200 \t 25.85 \t [1.0, -0.02, -0.03, 0.01, 0.04]\n",
      "1250 \t 62.91 \t [0.19, -0.36, -0.92, 0.04, 0.05]\n",
      "1300 \t 11.84 \t [0.94, -0.08, -0.01, 0.15, 0.01]\n",
      "1350 \t 14.49 \t [0.73, -0.01, -0.0, 0.25, 0.04]\n",
      "1400 \t 10.32 \t [0.89, -0.01, -0.04, 0.01, 0.16]\n",
      "1450 \t 32.49 \t [0.9, 0.02, -0.02, 0.01, 0.09]\n",
      "1500 \t 24.28 \t [1.06, -0.03, -0.03, 0.01, -0.0]\n",
      "1550 \t 19.34 \t [1.04, -0.04, -0.01, 0.01, -0.01]\n",
      "1600 \t 18.09 \t [1.0, -0.02, -0.01, 0.02, 0.0]\n",
      "1650 \t 18.67 \t [1.0, -0.03, -0.01, 0.03, 0.01]\n",
      "1700 \t 10.39 \t [0.72, -0.0, -0.01, 0.28, 0.01]\n",
      "1750 \t 17.19 \t [0.94, 0.03, -0.01, 0.04, 0.01]\n",
      "1800 \t 18.39 \t [0.98, -0.0, -0.02, 0.02, 0.03]\n",
      "1850 \t 15.03 \t [1.05, -0.03, -0.08, 0.01, 0.05]\n",
      "1900 \t 11.37 \t [1.12, 0.0, -0.19, 0.04, 0.02]\n",
      "1950 \t 22.71 \t [0.38, -0.01, -0.04, 0.04, 0.63]\n",
      "2000 \t 50.01 \t [0.99, -0.13, -2.59, 0.03, 0.7]\n",
      "2050 \t 146.26 \t [0.6, -2.38, 0.18, 0.4, 0.2]\n",
      "2100 \t 88.1 \t [1.55, -4.22, 0.59, 0.05, 1.03]\n",
      "2150 \t 53.05 \t [0.05, -0.78, 0.02, 1.66, 0.04]\n",
      "2200 \t 36.75 \t [0.1, -0.04, 0.02, 0.9, 0.03]\n",
      "2250 \t 30.9 \t [0.9, -0.01, 0.0, 0.1, 0.01]\n",
      "2300 \t 10.43 \t [0.43, -0.02, 0.0, 0.56, 0.02]\n",
      "2350 \t 14.35 \t [0.96, -0.03, -0.01, 0.01, 0.06]\n",
      "2400 \t 17.45 \t [0.99, -0.03, -0.0, 0.02, 0.02]\n",
      "2450 \t 15.1 \t [0.8, -0.01, 0.01, 0.18, 0.01]\n",
      "2500 \t 12.33 \t [0.92, -0.01, -0.0, 0.07, 0.02]\n"
     ]
    }
   ],
   "source": [
    "#  Short sales PERMITTED but floor is set to -10 and level to -10.\n",
    "\n",
    "for day in range(100, len(prices), 50):\n",
    "    prtf = boltzportfolio(prices[day-100:day], temp=55, floor=-10, level=-10, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( day, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The triple-digit geometric mean rates obviously\n",
    "cannot be taken at face value (RuntimeWarning issued\n",
    "but our program works around the mathematical limitation).\n",
    "They indicate extreme *pathological* conditions in the markets\n",
    "which will require special handling.\n",
    "They signal that a regime change is taking place,\n",
    "and that trading strategies must adapt quickly.\n",
    "See Appendix 1 for the detailed breakdown,\n",
    "including a narrative of leptokurtotic events.\n",
    "\n",
    "Such distress signals will *disappear* when short positions\n",
    "are restricted, as shown in the next cell,\n",
    "but the painful consequence is very negative portofolio rates!\n",
    "In other words, large drawdowns in wealth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100 \t 30.64 \t [0.13, 0.0, 0.0, 0.81, 0.05]\n",
      "150 \t 54.14 \t [0.06, 0.0, 0.0, 0.88, 0.06]\n",
      "200 \t 21.32 \t [0.03, 0.0, 0.0, 0.89, 0.08]\n",
      "250 \t -11.53 \t [0.0, 0.0, 0.12, 0.66, 0.22]\n",
      "300 \t -22.92 \t [0.01, 0.0, 0.0, 0.87, 0.12]\n",
      "350 \t -83.95 \t [0.02, 0.0, 0.0, 0.5, 0.49]\n",
      "400 \t 13.67 \t [0.0, 0.0, 0.0, 0.96, 0.04]\n",
      "450 \t 19.38 \t [0.23, 0.0, 0.0, 0.75, 0.02]\n",
      "500 \t 45.61 \t [0.91, 0.0, 0.0, 0.06, 0.03]\n",
      "550 \t 36.19 \t [0.86, 0.0, 0.0, 0.12, 0.03]\n",
      "600 \t 36.56 \t [0.93, 0.0, 0.0, 0.05, 0.02]\n",
      "650 \t 9.3 \t [0.93, 0.0, 0.0, 0.05, 0.01]\n",
      "700 \t 12.62 \t [0.93, 0.0, 0.0, 0.02, 0.04]\n",
      "750 \t 15.66 \t [0.08, 0.0, 0.0, 0.87, 0.06]\n",
      "800 \t 13.64 \t [0.09, 0.0, 0.0, 0.85, 0.06]\n",
      "850 \t 34.62 \t [0.91, 0.0, 0.0, 0.08, 0.01]\n",
      "900 \t 39.86 \t [0.98, 0.0, 0.0, 0.02, 0.0]\n",
      "950 \t 25.54 \t [0.92, 0.0, 0.0, 0.08, 0.0]\n",
      "1000 \t 28.62 \t [0.11, 0.0, 0.0, 0.89, 0.0]\n",
      "1050 \t 22.33 \t [0.01, 0.0, 0.0, 0.9, 0.09]\n",
      "1100 \t 16.79 \t [0.06, 0.0, 0.0, 0.9, 0.03]\n",
      "1150 \t 42.22 \t [0.91, 0.0, 0.0, 0.04, 0.05]\n",
      "1200 \t 24.61 \t [0.95, 0.0, 0.0, 0.01, 0.04]\n",
      "1250 \t -9.49 \t [0.78, 0.0, 0.0, 0.09, 0.13]\n",
      "1300 \t 10.3 \t [0.85, 0.0, 0.0, 0.14, 0.01]\n",
      "1350 \t 14.77 \t [0.78, 0.0, 0.0, 0.21, 0.01]\n",
      "1400 \t 10.72 \t [0.85, 0.0, 0.0, 0.0, 0.15]\n",
      "1450 \t 32.77 \t [0.89, 0.02, 0.0, 0.0, 0.09]\n",
      "1500 \t 22.18 \t [1.0, 0.0, 0.0, 0.0, 0.0]\n",
      "1550 \t 18.46 \t [0.99, 0.0, 0.0, 0.01, 0.0]\n",
      "1600 \t 18.36 \t [0.98, 0.0, 0.0, 0.02, 0.0]\n",
      "1650 \t 17.38 \t [0.95, 0.0, 0.0, 0.03, 0.01]\n",
      "1700 \t 10.57 \t [0.73, 0.0, 0.0, 0.27, 0.0]\n",
      "1750 \t 16.96 \t [0.93, 0.03, 0.0, 0.04, 0.0]\n",
      "1800 \t 17.59 \t [0.95, 0.0, 0.0, 0.02, 0.03]\n",
      "1850 \t 12.14 \t [0.94, 0.0, 0.0, 0.0, 0.06]\n",
      "1900 \t 6.08 \t [0.91, 0.0, 0.0, 0.06, 0.03]\n",
      "1950 \t 22.24 \t [0.35, 0.0, 0.0, 0.03, 0.61]\n",
      "2000 \t 8.14 \t [0.55, 0.0, 0.0, 0.01, 0.44]\n",
      "2050 \t -24.0 \t [0.41, 0.0, 0.29, 0.03, 0.27]\n",
      "2100 \t 2.78 \t [0.53, 0.0, 0.16, 0.0, 0.31]\n",
      "2150 \t 8.63 \t [0.03, 0.0, 0.01, 0.94, 0.02]\n",
      "2200 \t 35.41 \t [0.09, 0.0, 0.02, 0.87, 0.03]\n",
      "2250 \t 31.24 \t [0.91, 0.0, 0.0, 0.09, 0.0]\n",
      "2300 \t 10.91 \t [0.4, 0.0, 0.0, 0.59, 0.01]\n",
      "2350 \t 14.65 \t [0.94, 0.0, 0.0, 0.0, 0.06]\n",
      "2400 \t 17.29 \t [0.96, 0.0, 0.0, 0.02, 0.02]\n",
      "2450 \t 14.92 \t [0.84, 0.0, 0.0, 0.16, 0.0]\n",
      "2500 \t 12.73 \t [0.93, 0.0, 0.0, 0.05, 0.02]\n"
     ]
    }
   ],
   "source": [
    "#  Short sales is RESTRICTED because floor is set to 0.01 and level to 0.\n",
    "#  We want to avoid tiny long positions.\n",
    "\n",
    "for day in range(100, len(prices), 50):\n",
    "    prtf = boltzportfolio(prices[day-100:day], temp=55, floor=0.01, level=0, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( day, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For long-only portfolios, Emerging and Europe were rarely included\n",
    "in the portfolio. It was mostly a switching game between\n",
    "America and Gold at major turning points.\n",
    "Such shifts can be seen to occur more gradually if the\n",
    "periods are more overlapping --\n",
    "in the way prices are monitored in real-time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Closing remarks\n",
    "\n",
    "We shall cover sequential decisions in another notebook\n",
    "because portfolio weights alone do not dicate optimal actions.\n",
    "Here in Part 2 we observed variations in softmax temperature\n",
    "and the evolution of portfolio weights over time.\n",
    "\n",
    "Blindly following the weights is not optimal since there\n",
    "are rebalancing costs to consider.\n",
    "A long-only portfolio should also monitor long-short signals\n",
    "to prevent being crushed in extreme market conditions.\n",
    "See Appendix 1 regarding learning from stress conditions.\n",
    "\n",
    "Here we considered an equities portfolio but did not\n",
    "address optimal bet size relative to wealth and how the\n",
    "information set generated by the Boltzmann weights\n",
    "can sharpen decisions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Appendix 1: Portfolio weights under stress conditions\n",
    "\n",
    "We take a closer look at the global crisis during August 2011\n",
    "when stock markets globally became highly volatile and leptokurtotic.\n",
    "The Boltzmann portfolios recorded triple-digit\n",
    "geometric mean rates by leveraged long and short positions.\n",
    "\n",
    "The covariance matrix, and hence the pweights, are estimated\n",
    "within a window of 100 trading days, and advanced one day\n",
    "to obtain overlapping (\"rolling\") results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000 \t 40.48 \t [0.1, -0.04, -0.19, 1.12, 0.01]\n",
      "1001 \t 33.71 \t [0.15, -0.03, -0.13, 1.01, 0.01]\n",
      "1002 \t 30.47 \t [0.2, -0.02, -0.12, 0.94, 0.01]\n",
      "1003 \t 37.78 \t [0.16, -0.03, -0.14, 1.01, 0.01]\n",
      "1004 \t 32.26 \t [0.17, -0.02, -0.14, 0.98, 0.01]\n",
      "1005 \t 33.93 \t [0.15, -0.03, -0.13, 1.01, 0.01]\n",
      "1006 \t 19.03 \t [0.41, -0.01, -0.04, 0.64, 0.0]\n",
      "1007 \t 21.23 \t [0.37, -0.01, -0.05, 0.68, 0.0]\n",
      "1008 \t 32.29 \t [0.19, -0.03, -0.1, 0.94, 0.01]\n",
      "1009 \t 28.76 \t [0.21, -0.03, -0.08, 0.89, 0.01]\n",
      "1010 \t 32.77 \t [0.14, -0.03, -0.15, 1.03, 0.01]\n",
      "1011 \t 38.89 \t [0.1, -0.03, -0.22, 1.14, 0.01]\n",
      "1012 \t 40.02 \t [0.09, -0.03, -0.2, 1.13, 0.01]\n",
      "1013 \t 39.56 \t [0.13, -0.01, -0.18, 1.06, 0.01]\n",
      "1014 \t 62.03 \t [0.07, -0.02, -0.35, 1.28, 0.02]\n",
      "1015 \t 60.1 \t [0.08, -0.02, -0.37, 1.28, 0.04]\n",
      "1016 \t 86.64 \t [0.04, -0.03, -0.5, 1.47, 0.01]\n",
      "1017 \t 52.0 \t [0.06, -0.03, -0.14, 1.1, 0.02]\n",
      "1018 \t 114.82 \t [0.02, -0.08, -0.46, 1.49, 0.02]\n",
      "1019 \t 74.55 \t [0.04, -0.06, -0.21, 1.21, 0.01]\n",
      "1020 \t 133.66 \t [0.02, -0.1, -0.43, 1.49, 0.02]\n",
      "1021 \t 98.97 \t [0.03, -0.08, -0.31, 1.34, 0.02]\n",
      "1022 \t 81.36 \t [0.03, -0.09, -0.22, 1.27, 0.01]\n",
      "1023 \t 81.19 \t [0.04, -0.07, -0.2, 1.22, 0.02]\n",
      "1024 \t 87.07 \t [0.03, -0.08, -0.2, 1.23, 0.02]\n",
      "1025 \t 85.74 \t [0.04, -0.07, -0.18, 1.2, 0.02]\n",
      "1026 \t 108.53 \t [0.02, -0.09, -0.23, 1.28, 0.02]\n",
      "1027 \t 114.65 \t [0.02, -0.1, -0.24, 1.29, 0.02]\n",
      "1028 \t 126.27 \t [0.02, -0.09, -0.27, 1.32, 0.02]\n",
      "1029 \t 147.41 \t [0.02, -0.13, -0.46, 1.54, 0.02]\n",
      "1030 \t 275.77 \t [0.02, -0.47, -1.14, 2.55, 0.03]\n",
      "1031 \t 372.4 \t [0.02, -0.69, -1.6, 3.25, 0.03]\n",
      "1032 \t 235.83 \t [0.02, -0.42, -0.82, 2.2, 0.02]\n",
      "1033 \t 244.3 \t [0.02, -0.4, -1.03, 2.39, 0.02]\n",
      "1034 \t 247.01 \t [0.03, -0.44, -0.93, 2.31, 0.03]\n",
      "1035 \t 180.44 \t [0.03, -0.22, -0.7, 1.86, 0.03]\n",
      "1036 \t 173.45 \t [0.03, -0.15, -0.68, 1.77, 0.03]\n",
      "1037 \t 171.62 \t [0.03, -0.18, -0.55, 1.67, 0.03]\n",
      "1038 \t 173.97 \t [0.02, -0.23, -0.55, 1.73, 0.03]\n",
      "1039 \t 312.44 \t [0.03, -0.39, -1.52, 2.83, 0.05]\n",
      "1040 \t 264.41 \t [0.03, -0.28, -1.19, 2.4, 0.04]\n",
      "1041 \t 850.88 \t [0.06, -1.43, -4.15, 6.37, 0.15]\n",
      "1042 \t 582.45 \t [0.07, -1.15, -3.16, 3.11, 0.13]\n",
      "1043 \t 1016.12 \t [0.11, -1.63, -5.51, 5.83, 0.2]\n",
      "1044 \t 1398.99 \t [0.16, -2.35, -8.02, 8.95, 0.27]\n",
      "1045 \t 639.9 \t [0.08, -1.22, -3.83, 3.83, 0.13]\n",
      "1046 \t 758.73 \t [0.07, -1.83, -4.24, 4.86, 0.14]\n",
      "1047 \t 207.46 \t [0.02, -0.76, -1.04, 0.73, 0.04]\n",
      "1048 \t 215.95 \t [0.02, -0.72, -1.11, 0.76, 0.05]\n",
      "1049 \t 175.24 \t [0.01, -0.82, -0.66, 0.43, 0.04]\n",
      "1050 \t 149.63 \t [0.02, -0.96, -0.37, 0.27, 0.05]\n",
      "1051 \t 143.73 \t [0.02, -0.62, -0.65, 0.2, 0.05]\n",
      "1052 \t 136.41 \t [0.02, -0.86, -0.49, 0.27, 0.05]\n",
      "1053 \t 131.48 \t [0.02, -0.75, -0.57, 0.26, 0.04]\n",
      "1054 \t 126.26 \t [0.03, -0.85, -0.42, 0.19, 0.06]\n",
      "1055 \t 127.15 \t [0.02, -0.85, -0.43, 0.19, 0.06]\n",
      "1056 \t 130.99 \t [0.02, -0.85, -0.46, 0.22, 0.07]\n",
      "1057 \t 141.74 \t [0.01, -0.72, -0.57, 0.22, 0.05]\n",
      "1058 \t 132.49 \t [0.02, -0.82, -0.49, 0.23, 0.07]\n",
      "1059 \t 130.09 \t [0.02, -0.81, -0.48, 0.21, 0.06]\n",
      "1060 \t 126.43 \t [0.02, -0.78, -0.51, 0.22, 0.05]\n",
      "1061 \t 129.42 \t [0.02, -0.79, -0.49, 0.21, 0.06]\n",
      "1062 \t 118.24 \t [0.02, -0.62, -0.62, 0.18, 0.05]\n",
      "1063 \t 115.58 \t [0.02, -0.57, -0.67, 0.17, 0.05]\n",
      "1064 \t 113.31 \t [0.03, -0.49, -0.87, 0.25, 0.08]\n",
      "1065 \t 111.63 \t [0.03, -0.51, -0.79, 0.2, 0.07]\n",
      "1066 \t 110.0 \t [0.04, -0.49, -0.84, 0.22, 0.07]\n",
      "1067 \t 117.68 \t [0.02, -0.56, -0.7, 0.18, 0.05]\n",
      "1068 \t 113.01 \t [0.03, -0.52, -0.74, 0.17, 0.06]\n",
      "1069 \t 113.44 \t [0.02, -0.6, -0.59, 0.13, 0.04]\n",
      "1070 \t 110.88 \t [0.03, -0.59, -0.58, 0.1, 0.04]\n",
      "1071 \t 107.99 \t [0.03, -0.56, -0.62, 0.11, 0.04]\n",
      "1072 \t 110.2 \t [0.07, -0.42, -0.84, 0.14, 0.06]\n",
      "1073 \t 113.9 \t [0.06, -0.44, -0.85, 0.17, 0.06]\n",
      "1074 \t 115.88 \t [0.07, -0.42, -0.9, 0.19, 0.05]\n",
      "1075 \t 100.82 \t [0.08, -0.39, -1.11, 0.36, 0.06]\n",
      "1076 \t 110.23 \t [0.1, -0.37, -1.1, 0.31, 0.06]\n",
      "1077 \t 123.21 \t [0.12, -0.34, -1.11, 0.28, 0.05]\n",
      "1078 \t 135.24 \t [0.09, -0.34, -1.09, 0.29, 0.05]\n",
      "1079 \t 138.17 \t [0.1, -0.34, -1.13, 0.32, 0.05]\n",
      "1080 \t 134.31 \t [0.15, -0.29, -1.31, 0.4, 0.05]\n",
      "1081 \t 142.07 \t [0.15, -0.27, -1.3, 0.36, 0.06]\n",
      "1082 \t 154.78 \t [0.17, -0.24, -1.37, 0.37, 0.06]\n",
      "1083 \t 149.09 \t [0.2, -0.24, -1.4, 0.38, 0.06]\n",
      "1084 \t 132.52 \t [0.08, -0.35, -1.04, 0.27, 0.05]\n",
      "1085 \t 132.52 \t [0.11, -0.31, -1.11, 0.26, 0.05]\n",
      "1086 \t 142.55 \t [0.14, -0.3, -1.27, 0.38, 0.05]\n",
      "1087 \t 149.19 \t [0.13, -0.36, -1.24, 0.42, 0.04]\n",
      "1088 \t 153.97 \t [0.12, -0.36, -1.22, 0.41, 0.04]\n",
      "1089 \t 134.15 \t [0.06, -0.48, -0.92, 0.3, 0.03]\n",
      "1090 \t 116.75 \t [0.03, -0.59, -0.64, 0.17, 0.02]\n",
      "1091 \t 120.03 \t [0.03, -0.61, -0.63, 0.19, 0.02]\n",
      "1092 \t 123.69 \t [0.03, -0.71, -0.51, 0.17, 0.03]\n",
      "1093 \t 121.54 \t [0.02, -0.68, -0.53, 0.16, 0.03]\n",
      "1094 \t 122.28 \t [0.02, -0.7, -0.52, 0.16, 0.03]\n",
      "1095 \t 128.12 \t [0.02, -0.6, -0.6, 0.14, 0.03]\n",
      "1096 \t 115.67 \t [0.02, -0.62, -0.56, 0.13, 0.02]\n",
      "1097 \t 112.79 \t [0.03, -0.52, -0.67, 0.13, 0.03]\n",
      "1098 \t 97.22 \t [0.05, -0.47, -0.78, 0.17, 0.03]\n",
      "1099 \t 103.71 \t [0.06, -0.48, -0.75, 0.15, 0.03]\n",
      "1100 \t 102.07 \t [0.06, -0.46, -0.78, 0.15, 0.03]\n"
     ]
    }
   ],
   "source": [
    "#  Short sales PERMITTED but floor is set to -10 and level to -10.\n",
    "\n",
    "for day in range(1000, 1101, 1):\n",
    "    prtf = boltzportfolio(prices[day-100:day], temp=55, floor=-10, level=-10, n=2)\n",
    "    pweights = [ w for w, r, k in prtf[1] ]\n",
    "    print( day, \"\\t\", prtf[0], \"\\t\", pweights )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "            America  Emerging  Europe    Gold  Japan\n",
       "T                                                   \n",
       "2011-07-28   130.16     46.91   35.89  157.32  42.60\n",
       "2011-07-29   129.05     47.15   35.90  158.41  42.84\n",
       "2011-08-01   128.71     47.07   35.02  157.70  42.40\n",
       "2011-08-02   125.49     45.62   33.63  161.52  42.08\n",
       "2011-08-03   126.21     45.47   33.97  161.41  42.16\n",
       "2011-08-04   120.18     42.79   31.36  160.64  39.44\n",
       "2011-08-05   119.98     42.57   32.47  161.78  39.96\n",
       "2011-08-08   112.26     39.08   29.48  167.12  38.04\n",
       "2011-08-09   117.37     41.26   31.31  168.61  40.00\n",
       "2011-08-10   112.35     39.13   28.98  174.58  38.08\n",
       "2011-08-11   117.28     41.32   30.36  170.67  39.60\n",
       "2011-08-12   118.15     41.26   31.30  169.97  39.28\n",
       "2011-08-15   120.55     42.23   32.24  171.80  39.72\n",
       "2011-08-16   119.56     41.84   31.52  173.92  39.40\n",
       "2011-08-17   119.67     42.17   31.62  174.42  39.44\n",
       "2011-08-18   114.61     40.16   29.91  177.73  38.12\n",
       "2011-08-19   112.68     39.73   29.17  179.99  37.72\n",
       "2011-08-22   112.73     39.63   29.47  184.59  37.80\n",
       "2011-08-23   116.46     41.06   30.51  177.60  38.56\n",
       "2011-08-24   118.13     40.62   30.72  171.63  38.08\n",
       "2011-08-25   116.31     39.83   29.92  172.23  37.56\n",
       "2011-08-26   118.04     40.52   30.27  177.49  38.36\n",
       "2011-08-29   121.38     41.81   31.12  173.89  38.88\n",
       "2011-08-30   121.62     41.91   30.91  179.05  38.72\n",
       "2011-08-31   122.06     42.72   31.52  177.65  39.48\n",
       "2011-09-01   120.96     42.52   30.92  177.87  39.04\n",
       "2011-09-02   117.84     41.56   29.86  183.23  38.32\n",
       "2011-09-06   116.99     41.14   28.41  182.92  37.48\n",
       "2011-09-07   120.27     42.38   29.43  177.08  38.28\n",
       "2011-09-08   119.02     41.39   28.70  181.78  37.76\n",
       "2011-09-09   115.86     40.00   27.19  180.70  36.92\n",
       "2011-09-12   116.66     39.93   26.78  176.67  37.28\n",
       "2011-09-13   117.58     40.01   27.12  178.54  37.92\n",
       "2011-09-14   119.36     39.91   28.02  177.21  38.08\n",
       "2011-09-15   121.47     40.43   29.04  174.39  38.60\n",
       "2011-09-16   121.58     40.53   28.91  176.03  38.96\n",
       "2011-09-19   120.31     39.31   27.93  173.27  38.28\n",
       "2011-09-20   120.17     39.04   27.99  175.78  38.32\n",
       "2011-09-21   116.51     37.60   27.12  173.59  37.84\n",
       "2011-09-22   112.80     34.95   26.07  169.05  37.20\n",
       "2011-09-23   113.54     35.88   26.40  159.80  37.36\n",
       "2011-09-26   116.25     36.42   27.22  157.58  37.56\n",
       "2011-09-27   117.60     37.58   28.17  160.63  38.16\n",
       "2011-09-28   115.16     36.43   27.69  156.20  38.20\n",
       "2011-09-29   116.02     36.95   28.57  157.70  38.84\n",
       "2011-09-30   113.17     35.07   27.32  158.08  37.84\n",
       "2011-10-03   109.93     34.36   26.16  160.99  37.24\n",
       "2011-10-04   112.13     34.88   26.86  157.63  37.72\n",
       "2011-10-05   114.50     35.72   27.56  159.49  37.52\n",
       "2011-10-06   116.51     36.88   28.57  160.49  37.72"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Investigate the leptokurtotic prices:\n",
    "prices[1010:1060]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### August 2011 events: \n",
    "\n",
    "- ***Black Monday 2011*** refers to 8 August 2011, when stock markets worldwide crashed following the Friday night credit rating downgrade by Standard and Poor's of the **United States sovereign debt** from AAA, or \"risk free\", to AA+. It was the first time in history the United States was downgraded.\n",
    "\n",
    "- United States: On 8 August 2011, the S&P 500 lost 79.92 points (6.7%) to 1,119.46 points with all 500 stocks falling.\n",
    "\n",
    "- On 11 August 2011, the market authorities of Belgium, Italy, France and Spain as well as the European financial regulator ESMA announced the *ban of all forms of short selling on banks and other financial companies* as a result of growing instability in markets on rumours of French banks risking downgrades and concerns of various European banks that are highly exposed to indebted nations such as Greece.\n",
    "\n",
    "- France: The CAC40 fell by 20% in two weeks. It fell by 5.5% on 18 August 2011. The CAC thus moved from 3,800 points to 3,000 points in 10 days.\n",
    "\n",
    "- Germany: The DAX fell by 5.8% on 18 August 2011.\n",
    "\n",
    "Source: https://en.wikipedia.org/wiki/August_2011_stock_markets_fall\n",
    "\n",
    "- **Physical gold** (not the ETF GLD) at 1623 USD on 1 August 2011, increased to 1878 USD by 22 August 2011.\n",
    "\n",
    "Source: https://fred.stlouisfed.org/series/GOLDPMGBD228NLBM\n",
    "\n",
    "The Boltzmann portfolio was mostly long Gold just prior to the crisis,\n",
    "which in retrospect performed most favorably.\n",
    "The portfolio towards the end of August 2011 shifts to shorting Emerging."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning from stress conditions\n",
    "\n",
    "Stress on a system is valuable in understanding\n",
    "how the components parts can be improved\n",
    "for sequential decision-making in production.\n",
    "\n",
    "We shall consider how extreme signals\n",
    "can be used to generate contrarian decisions in the Part 3.\n",
    "Excess greed or fear in the markets,\n",
    "observed as bubbles or anti-bubbles, invite such decisions.\n",
    "\n",
    "A portfolio manager may seek *less volatile weights*.\n",
    "This can be achieved in a number of ways:\n",
    "effectively lengthen the sample size for computing\n",
    "the covariance matrix, increase softmax temperature,\n",
    "apply Kalman filtering on the weights themselves\n",
    "which could be predictive to further help optimize\n",
    "portfolio returns across multiple rebalancing rules."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "- On softmax:\n",
    "    - https://en.wikipedia.org/wiki/Softmax_function\n",
    "    - https://compute.quora.com/What-is-softmax\n",
    "    - http://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative\n",
    "    - http://cs231n.github.io/linear-classify/#softmax\n",
    "    - https://en.wikipedia.org/wiki/Reinforcement_learning"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}