quantum/randquantum-demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# True randomness: demo of randquantum module \n",
    "\n",
    "This notebook gives a demonstration of the useful functions which yield *true* random numbers produced live from an experiment in quantum mechanics.  They are contained in the **randquantum** Python module under the **randomsys** project: https://git.io/randomsys \n",
    "\n",
    "Reliable and unbiased random numbers are needed for a range of applications from numerical modeling to cryptographic communications. While there are algorithms that can generate pseudo random numbers, they can never be perfectly random nor indeterministic. ANU researchers are generating *true random numbers from a physical quantum source* by splitting a beam of light into two beams and then measuring the power in each beam. Because light is quantised, the light intensity in each beam fluctuates about the mean. Those fluctuations, due ultimately to the quantum vacuum, can be converted into a source of random numbers. \n",
    "\n",
    "The rate at which the live bits are streamed is limited by the bandwidth of your internet connection. Every number is randomly generated in real time and cannot be predicted beforehand. Most pseudo-random numbers have a finite period after which the sequence repeats. The output herein will not have such periodicity.\n",
    "\n",
    "Details and references can be found in the Appendix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*CHANGE LOG*\n",
    "\n",
    "    2015-10-10  Add section on \"Question of independence\" and mention unittest.\n",
    "    2015-10-05  First draft."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " ::  TIMESTAMP of last notebook execution:\n",
      "Sat Oct 10 11:59:14 PDT 2015\n",
      " ::  IPython version:\n",
      "3.2.1\n",
      " ::  Working directory (set as $workd):\n",
      "/home/yaya/Dropbox/share/git/nous/randomsys/quantum\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#  NOTEBOOK v4 settings and system details:      [00-tpl v4.15.0812]\n",
    "\n",
    "#  Assume that the backend is LINUX (e.g. Ubuntu running bash shell):\n",
    "print '\\n ::  TIMESTAMP of last notebook execution:'\n",
    "!date\n",
    "print ' ::  IPython version:'\n",
    "!ipython --version\n",
    "\n",
    "#  Automatically RELOAD modified modules:\n",
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "#           0 disables autoreload.\n",
    "\n",
    "#  DISPLAY options\n",
    "from IPython.display import Image \n",
    "#  e.g. Image(filename='holt-winters-equations.png', embed=True) # url= also works\n",
    "from IPython.display import YouTubeVideo\n",
    "#  e.g. YouTubeVideo('1j_HxD4iLn8', start='43', width=600, height=400)\n",
    "from IPython.display import HTML # useful for snippets\n",
    "#  e.g. HTML('<iframe src=http://en.mobile.wikipedia.org/?useformat=mobile width=700 height=350></iframe>')\n",
    "from IPython.core import page\n",
    "get_ipython().set_hook('show_in_pager', page.as_hook(page.display_page), 0)\n",
    "#  Or equivalently in config file: \"InteractiveShell.display_page = True\", \n",
    "#  which will display results in secondary notebook pager frame in a cell.\n",
    "\n",
    "#  MATH display, use %%latex, rather than the following:\n",
    "#                from IPython.display import Math\n",
    "#                from IPython.display import Latex\n",
    "#  Generate PLOTS inside notebook:\n",
    "%matplotlib inline\n",
    "\n",
    "print ' ::  Working directory (set as $workd):'\n",
    "workd, = !pwd\n",
    "print workd + '\\n'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import randquantum as rq\n",
    "import numpy as np   #  numpy is not a dependency, just here for stats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Truly random seed \n",
    "\n",
    "Most pseudo-random number generators require a initial starting point called a *seed*. It should obviously be random, and is usually derived from the computer current state such as time in milliseconds. The randquantum module by design does not need any seed -- in fact, it is a reliable source to **generate a random integer of any arbitrary length** (constrained only by memory size), suitable as a seed for other projects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7980363771586135570665388004491861862596096991061075686025032257083757164391\n"
     ]
    }
   ],
   "source": [
    "#  Demo random integer of length 76:\n",
    "print rq.seed( 76 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The implication is that the outputs from randquantum are not replicable! Try it out in this IPython notebook (download and execute locally).\n",
    "\n",
    "## Random digits and statistical testing\n",
    "\n",
    "Here one call to rq.nine() will yield a single random digit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rq.nine()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now informally see the distribution across ten digits 0 through 9. It should look uniform, otherwise there is bias and this entire project becomes invalid. \n",
    "\n",
    "The formal tests for our *true* random numbers can be found at: http://qrng.anu.edu.au/NIST.php -- daily testing results are posted there. Within this project, we have a unittest module called *test_randquantum.py*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1003, 1025, 1004, 1002, 991, 1014, 994, 986, 1016, 965]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "counter = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n",
    "for i in range( 10000 ):\n",
    "    d = rq.nine()\n",
    "    counter[d] += 1\n",
    "    \n",
    "counter    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generating ten thousand samples, the count for each digit is roughly one thousand, so informally we can say, \"not bad.\" The implicit point is that *you can place a randquantum **generator** function anywhere in your source code to create true random conditions indefintely.*\n",
    "\n",
    "The online source of randomness supplies us with a list of *finite* length, but we overcome this API limitation by writing generators. The online dependency is somewhat problematic for offline projects so we have a simulated fallback, just in case (and warnings to stderr will be issued)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question of independence\n",
    "\n",
    "One might wonder if our random data points are indeed independent. By the theory of quantum mechanics, the authentic points should be independent *if* the underlying experiment has been implemented correctly. Our fallback pseudo points have been extensively tested for statistical independence. \n",
    "\n",
    "We *induce independence by creating a hybrid* between authentic\n",
    "and pseudo. Since the sources are clearly independent, this method also\n",
    "stochastically disrupts the deterministic periodicity of pseudo generation.\n",
    "Also by incorporating pseudo, we fetch fewer times from the server, \n",
    "increasing overall execution speed.\n",
    "\n",
    "In effect, we inject authentic entropy into a chaotic system with maximum uncertainty \n",
    "as to the random source. See randquantum() for details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#  Uncomment next line for said details:\n",
    "#  rq.randquantum??"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Arbitrarily large random integers and permutations\n",
    "\n",
    "Even for small len(x), the total number of permutations of x is \n",
    "larger than the period of most PSEUDO random number generators; \n",
    "this implies that most permutations of a long sequence can \n",
    "never be generated. But this is not true for randquantum_authentic;\n",
    "though if the listing is huge, this could take a long time to finish."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "39514481746644540999L"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Truly find a random integer between 0 and some arbitrary integer:\n",
    "rq.randint( 98765432109876543210 )\n",
    "#  L is for internal representation of long integer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given the function **randint** we can pick out random element(s) from any arbitrarily large list, with or without replacement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['rat', 'cat']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zoo = [ 'dog', 'cat', 'mice', 'rat', 'lion', 'tiger', 'snake']\n",
    "\n",
    "#  Pick out two randomly:\n",
    "rq.randpick( zoo, count=2, replace=False )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['tiger', 'snake', 'dog', 'rat', 'mice', 'lion', 'cat']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Now the difficult part in theory: repeatedly \n",
    "#  produce a true permutation of an arbitrarily large list:\n",
    "rq.shuffle( zoo )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If every atom in the universe has a distinct label at a moment in time, and we could successfully shuffle those labels repeatedly since our generators do not have periodicity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Randomness from a Gaussian distribution\n",
    "\n",
    "Having a true uniform distribution permits one to mathematically \n",
    "synthesize a Gaussian distribution. That happens in the generator \n",
    "function **gauss**. We use the Kinderman Ratio of Uniform Deviates method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-1.0073194937\n",
      "-0.5873444406\n",
      "1.37348191047\n",
      "-2.330064314\n",
      "-0.615299593167\n"
     ]
    }
   ],
   "source": [
    "#  Sample from Normal N(0, 1) distribution, five times:\n",
    "for i in range( 5 ):\n",
    "    print rq.gauss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let informally see if we are getting random samples from a N(0, 1). We the numpy module to compute the mean and standard deviation for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#  Insert a list into a numpy array:\n",
    "gauss_output = np.array( rq.gaussquantum( 10000, mean= 0, sdev=1.0))\n",
    "\n",
    "#  Module code shows gaussquantum returns a list which is used by gauss()."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000,)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  Verify sample size:\n",
    "np.shape( gauss_output )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0027329597923485597"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  numpy calculates the sample mean:\n",
    "gauss_output.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0082313924578179"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  numpy standard deviation method:\n",
    "gauss_output.std()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quintessential uniform real from [0, 1)\n",
    "\n",
    "The sample values are truly random, but they are not from a continous semi-open interval. \n",
    "FAQ: Why not?\n",
    "\n",
    "Answer:  These true random numbers originate from the **quantum** world \n",
    "where fundamentally everything is *discrete*. \n",
    "(Pseudo generators have a similar problem, but due to the limited \n",
    "numerical precision of floats which is hardware dependent.)\n",
    "\n",
    "Next, we demonstrate **real()**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.517921721212\n",
      "0.297108415348\n",
      "0.893171587692\n",
      "0.943831540389\n",
      "0.692317082468\n"
     ]
    }
   ],
   "source": [
    "#  Sample from real interval [0, 1) uniformly:\n",
    "for i in range( 5 ):\n",
    "    print rq.real()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Even more essential: boolean()\n",
    "\n",
    "Getting a truly random binary output is at the heart of our experiments. From this alone, we can in theory simulate the evolution of the entire universe. Let there be photons!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "0\n",
      "1\n",
      "0\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "#  Random boolean values:\n",
    "for i in range( 5 ):\n",
    "    print rq.boolean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lucky day!\n"
     ]
    }
   ],
   "source": [
    "#  Use it freely in conditionals:\n",
    "if rq.boolean():\n",
    "    print \"Lucky day!\"\n",
    "else:\n",
    "    print \"No pizza.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fun ready-made generators\n",
    "\n",
    "The randquantum module allows you to create your own custom random generator. For example, rq.trio() yields one of three equally possible: -1, 0, 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "1\n",
      "-1\n",
      "-1\n"
     ]
    }
   ],
   "source": [
    "#  Random trio values:\n",
    "for i in range( 5 ):\n",
    "    print rq.trio()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# WRAP-UP\n",
    "\n",
    "We demonstrated the following useful functions defined in the **randquantum** module:\n",
    "\n",
    "- seed()\n",
    "- nine()\n",
    "- randint()\n",
    "- randpick()\n",
    "- gauss()\n",
    "- real()\n",
    "- boolean()\n",
    "- trio()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Appendix 1: time executions\n",
    "\n",
    "IPython has a nice way to time Python functions and expressions. Wall time will vary according to your internet speed, thus this illustrates I/O bound versus CPU bound processes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 796 ms, sys: 20 ms, total: 816 ms\n",
      "Wall time: 5.72 s\n"
     ]
    }
   ],
   "source": [
    "%time for i in range( 10000 ): rq.nine()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Appendix 2: references and source code\n",
    "\n",
    "*So how exactly does the randomness get introduced into a computer program?* The physical source is described below, and the data transfer relies on the function getanu(). The rest is basically a matter of mathematics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " ::  THIS IS A MODULE for import -- not for direct execution! \n",
      "\n",
      "Enter something to get out: OK\n"
     ]
    }
   ],
   "source": [
    "# %load randquantum.py\n",
    "#  Python Module for import                           Date : 2015-10-10\n",
    "''' \n",
    "_______________|  randquantum.py : true random numbers using quantum mechanics. \n",
    "                      Repository : https://github.com/rsvp/randomsys\n",
    "\n",
    "     [For simple USAGE examples, skip to the conclusion at the end.]\n",
    "\n",
    "Reliable and unbiased random numbers are needed for a range of applications\n",
    "spanning from numerical modeling to cryptographic communications. While there\n",
    "are algorithms that can generate pseudo random numbers, they can never be\n",
    "perfectly random nor indeterministic.\n",
    "\n",
    "ANU researchers are generating true random numbers from a physical quantum\n",
    "source by splitting a beam of light into two beams and then measuring the\n",
    "power in each beam.  Because light is quantised, the light intensity in each\n",
    "beam fluctuates about the mean.  Those fluctuations, due ultimately to the\n",
    "quantum vacuum, can be converted into a source of random numbers. \n",
    "\n",
    "In classical physics, a vacuum is considered as a space that is empty of\n",
    "matter or photons. In quantum mechanics, however, that same space resembles a\n",
    "sea of virtual particles appearing and disappearing all the time. This result\n",
    "is due to the fact that the vacuum still possesses a zero-point energy.\n",
    "Consequently, the electromagnetic field of the vacuum exhibits random\n",
    "fluctuations in phase and amplitude at all frequencies. By measuring these\n",
    "fluctuations, one can generate ultra-high bandwidth random numbers.\n",
    "\n",
    "The raw output of a quantum random-number generator is usually tainted by\n",
    "classical technical noise. The integrity of the device can be compromised if\n",
    "this noise is tampered with or even controlled by some malicious party. To\n",
    "safeguard against this, our method produces side-information-independent\n",
    "randomness that is quantified by min-entropy conditioned on this classical\n",
    "noise. It maximizes the conditional min entropy of the number sequence\n",
    "generated from a given quantum-to-classical-noise ratio. The detected\n",
    "photocurrent in the experiment has a real-time random-number generation rate\n",
    "of 14 (Mbit/s)/MHz. The spectral response of the detection system shows the\n",
    "potential to deliver more than 70 Gbit/s of random numbers.\n",
    "\n",
    "The 2015 hardware is constantly generating random bits at a rate of 5.7\n",
    "Gbits/s.  The rate at which the live bits are streamed is limited by the\n",
    "bandwidth of the internet connection.  Every number is randomly generated in\n",
    "real time and cannot be predicted beforehand. \n",
    "\n",
    "Most pseudo random numbers have a finite period. Good pseudo random number\n",
    "generators (e.g. the Mersenne Twister) have humoungous periods. But\n",
    "eventually, if we wait long enough, the sequence will repeat itself. All\n",
    "pseudo generators are indeed deterministic.  In contrast, for our true random\n",
    "number generator, even if two exactly identical generators were placed in\n",
    "identical environments with identical initial conditions, the two streams of\n",
    "number generated will still be totally uncorrelated.\n",
    "\n",
    "In development, one can use pseudo random numbers for simulations (since their\n",
    "performance is not I/O bound) -- but as a final double-check on the results, \n",
    "use random numbers which are truly random.\n",
    "\n",
    "If the server is somehow inaccessible, this module is written to fallback \n",
    "on pseudo simulation of the authentic process (with warnings emitted).\n",
    "\n",
    "New 2015-10-09: we induce independence by creating a hybrid between authentic\n",
    "and pseudo. Since the sources are clearly independent, this method also\n",
    "stochastically disrupts the deterministic periodicity of pseudo generation.\n",
    "Also by incorporating pseudo, we fetch fewer times from the server, \n",
    "increasing speed.\n",
    "\n",
    "Rigorous statistical TEST RESULTS daily:  http://qrng.anu.edu.au/NIST.php\n",
    "Internal statistical tests uses unittest:  test_randquantum.py\n",
    "\n",
    "\n",
    "References:\n",
    "\n",
    "- Real time demonstration of high bitrate quantum random number generation\n",
    "  with coherent laser light T. Symul, S. M. Assad, and P. K. Lam:  \n",
    "  Appl. Phys.  Lett. 98, 231103 (2011)\n",
    "\n",
    "- Maximization of Extractable Randomness in a Quantum Random-Number Generator\n",
    "  J. Y. Haw, S. M. Assad, A. M. Lance, N. H. Y. Ng, V. Sharma, P. K. Lam, and\n",
    "  T. Symul: Phys. Rev. Applied 3, 054004 (2015)\n",
    "\n",
    "- Australia National University, ANU:\n",
    "  http://photonics.anu.edu.au/qoptics/Research/qrng.php\n",
    "\n",
    "- General resources and papers on randomness, including statistical testing:\n",
    "  http://qrng.anu.edu.au/Links.php#statistical_tests\n",
    "\n",
    "\n",
    "CHANGE LOG  Latest version available at https://git.io/randomsys\n",
    "2015-10-10  Add BOOLauthentic switch for debugging convenience.\n",
    "2015-10-09  Edit comments, getanu() and randquantum() docstrings for clarity.\n",
    "2015-10-08  Induce independence by hybrid between authentic and pseudo,\n",
    "               rename old randquantum() as randquantum_safe().\n",
    "               AUTH sets probability of authentic service.\n",
    "               Number the warnings issued by global Nwarn.\n",
    "               Remove default length argument.\n",
    "2015-10-05  Introduce NINERS variable for float representation.\n",
    "               Fix in-place .remove() bug in randpick by copy operation.\n",
    "2015-10-03  Change real() from [0, 1.0] to [0, 1.0) for compatibility.\n",
    "               Rewrite gaussquantum to use generators.\n",
    "               Use Python random.randrange as offline fallback.\n",
    "2015-10-02  Rename intquantum to b16quantum as a precaution.\n",
    "               Add randint to supplement b16quantum.\n",
    "               Rename a generator to nine. \n",
    "               Rewrite seed to use efficient generator.\n",
    "               Add randpick and permute [shuffle] for arbitrarily long list.\n",
    "2015-09-30  Add generator functionality for sipping stream indefinitely.\n",
    "2015-09-29  First version. \n",
    "'''\n",
    "\n",
    "import urllib2\n",
    "from math   import log\n",
    "\n",
    "from random import randrange as pseudorange \n",
    "from sys    import stderr                    #  Used to warn of fallback.\n",
    "\n",
    "AUTH = 0.50\n",
    "#      Non-zero prob(authentic), should be reciprocal of positive integer; \n",
    "#      see randquantum() which stochastically mixes in pseudo,\n",
    "#      inducing independence due to use of two sources.\n",
    "#      Recommend 0.50 for maximum uncertainty.\n",
    "#      For fully authentic output, let AUTH = 1.0 obviously.\n",
    "\n",
    "bestlen = int( 1024 / AUTH )\n",
    "#         Helps to minimize calls to server.\n",
    "\n",
    "BOOLauthentic = True\n",
    "#   Set to False to see only pseudo results. Overrides AUTH for debugging.\n",
    "\n",
    "NINERS = 0.99999999999\n",
    "#        ^reasonable system-dependent float representation of (1 - epsilon).\n",
    "#         More nines can cause rare unintended roundup errors. \n",
    "\n",
    "Nwarn = 0\n",
    "#       Number of warnings. Usually indicative of failed calls to server.\n",
    "\n",
    "def warn( message ):\n",
    "    '''Send warning message via stderr.'''\n",
    "    #  Portable for both Python 2 and 3.\n",
    "    global Nwarn\n",
    "    Nwarn += 1\n",
    "    stderr.write( ' :!  Warning #' + str(Nwarn) + ': '  + message + '\\n')\n",
    "\n",
    "\n",
    "def getanu( url='https://qrng.anu.edu.au/API/jsonI.php?length=1024&type=uint16' ):\n",
    "    '''Download list of Quantum Random Numbers from Australia National University.\n",
    "    Note: \"uint16\" returns integers between 0-65535 INCLUSIVE of endpoints, \n",
    "    and maximum length permitted is 1024 (but multiple calls are permitted).\n",
    "    Each time you download the \"live stream\" via the functions defined below,\n",
    "    the server will deliver new and unique random numbers. Moreover, these\n",
    "    pages are authenticated and SSL encrypted for security.\n",
    "\n",
    "    See API doc: https://qrng.anu.edu.au/API/api-demo.php\n",
    "        Contact: cqc2t@anu.edu.au\n",
    "    '''\n",
    "    #  print \"DEBUG: getanu() waiting for server to respond...\"\n",
    "    page = urllib2.urlopen( url, timeout=2 )\n",
    "    #  print \"DEBUG: server OK, retrieved json line.\"\n",
    "    # ----------------\n",
    "    json = page.read()\n",
    "    #  For length=3, json looks like:\n",
    "    #      {\"type\":\"uint16\",\"length\":3,\"data\":[7731,40732,1971],\"success\":true} \n",
    "    #  but we ignore the json module, and use brute force:\n",
    "    json_after = json.split('[')[1]\n",
    "    strlist = json_after.split(']')[0].split(',')\n",
    "    #  Above read as string, so of course, convert to integer for our list:\n",
    "    return [ int(s) for s in strlist ]\n",
    "\n",
    "\n",
    "def randquantum_authentic( length ):\n",
    "    '''Quantum random integers between [0, 65535] inclusive in a list.\n",
    "    The data is online thus the performance is I/O bound.\n",
    "    ''' \n",
    "    #  length of 1024 will make exactly one call to server.\n",
    "    #  We must possibly make multiple calls to overcome API length limitation.\n",
    "    calls = int(( length / 1024.5 ) + 1 )\n",
    "    biglist = []\n",
    "    for i in range( calls ):\n",
    "        biglist += getanu()\n",
    "    return biglist[:length]\n",
    "\n",
    "\n",
    "def randquantum_pseudo( length ): \n",
    "    '''Pseudo simulation of randquantum_authentic(), intended as fallback.\n",
    "    Offline call to the standard Python package random.\n",
    "    '''\n",
    "    return [ pseudorange(0, 65536) for i in range(length) ]\n",
    "\n",
    "\n",
    "def randquantum_safe( length, authentic=BOOLauthentic ):\n",
    "    '''Authentic PRIMARY DEPENDENCY with offline FALLBACK.\n",
    "    ___ATTN___  \"authentic\" switch for developer's debugging only.\n",
    "    '''\n",
    "    if authentic:\n",
    "        try:\n",
    "            #  Possible spend \"timeout\" seconds here in limbo...\n",
    "            return randquantum_authentic( length )\n",
    "        except:\n",
    "            warn(         \"randquantum_authentic FAIL: now, PSEUDO simulation.\" )\n",
    "            return randquantum_pseudo( length )\n",
    "    else:\n",
    "        warn( \"authentic=False for randquantum_safe implies PSEUDO simulation.\" )\n",
    "        return randquantum_pseudo( length )\n",
    "\n",
    "\n",
    "def randquantum( length ):\n",
    "    '''Induce INDEPENDENCE by HYBRID between authentic and pseudo.\n",
    "    The sources are clearly independent. This method also stochastically \n",
    "    disrupts the deterministic periodicity of pseudo generation.\n",
    "    SPEED: by incorporating pseudo, we fetch fewer times from the server.\n",
    "\n",
    "    We set aside an authentic list from which we will draw upon \n",
    "    with prob(authentic), otherwise we call upon some pseudo number, \n",
    "    and thus we grow the hybrid list to desired length.\n",
    "    '''\n",
    "    #            AUTH at the top sets prob(authentic).\n",
    "    aulen = int( AUTH * length )\n",
    "    safe  = randquantum_safe( aulen )  \n",
    "    #       ^authentic with fallback provision, which means hybrid \n",
    "    #       could be all pseudo if authentic fails entirely.\n",
    "    authinverse = int( 1 / AUTH )\n",
    "    hybrid = []\n",
    "    i = aulen - 1\n",
    "    while len(hybrid) < length:\n",
    "        if pseudorange( 0, authinverse ):\n",
    "            #              ^stochastically mixes authentic with pseudo.\n",
    "            hybrid.append( pseudorange(0, 65536) )\n",
    "        else:\n",
    "            #  Pick element from tailend of safe in reverse order.\n",
    "            if i >= 0:\n",
    "                hybrid.append( safe[i] )\n",
    "                i -= 1\n",
    "    return hybrid[:length]\n",
    "\n",
    "\n",
    "def boolquantum( length ):\n",
    "    '''Convert randquantum to a random list of zeros and ones.\n",
    "    In Python, 0 is False, anything else True, hence this is boolean.\n",
    "    '''\n",
    "    return [ i % 2 for i in randquantum( length ) ]\n",
    "\n",
    "\n",
    "def realquantum( length, endpoint=1.0 ):\n",
    "    '''Convert randquantum to random real numbers: [0, endpoint]\n",
    "    Discrete resolution is 1.52590219e-05 for [0,1].\n",
    "    '''\n",
    "    multiplier = float( endpoint ) / 65535 \n",
    "    return [ i * multiplier for i in randquantum( length ) ]\n",
    "\n",
    "\n",
    "def b16quantum( length, endinteger=9 ):\n",
    "    '''Random integers: [0, endinteger] where endinteger < 65536.\n",
    "    For larger endinterger, consult randint below.\n",
    "    If your endinteger is 1, we recommend boolquantum() instead.\n",
    "    '''\n",
    "    endpoint = endinteger + NINERS\n",
    "    return [ int(r) for r in realquantum( length, endpoint ) ]\n",
    "\n",
    "\n",
    "\n",
    "# ======================================================= GENERATORS =========== \n",
    "#  Above has focused on lists, but in practice, \n",
    "#  generators are more natural within other scripts.\n",
    "#  The user should not worry about extracting an element from a random list.\n",
    "#  Also these generators will cache the unused portion of a list in memory, \n",
    "#  and stand-by for a yield statement thus conserving calls to the server.\n",
    "#  The generators below will \"fill-up\" only when needed (so this means \n",
    "#  the quantity of data is not pre-set unlike the lists above).\n",
    "\n",
    "\n",
    "def sipstream( func_quantum, argtuple=(bestlen,) ):\n",
    "     '''Generalized generator for certain randquantum functions. \n",
    "     Consider a stream to be lists being downloaded.\n",
    "     This generator yields an element of a list as it is needed\n",
    "     iteratively. Usage example:\n",
    "          import randquantum as rq\n",
    "          sip = sipstream( rq.gaussquantum, (1024, 0, 1.0) )\n",
    "          #     where the tuple serves as positional arguments.\n",
    "          print next( sip )\n",
    "          print next( sip )\n",
    "          print next( sip )\n",
    "     '''\n",
    "     length = argtuple[0]\n",
    "     stream = apply( func_quantum, argtuple )\n",
    "     i = 0\n",
    "     while True:\n",
    "          yield stream[i]\n",
    "          i += 1\n",
    "          if i == length:\n",
    "               i = 0\n",
    "               #   And FRESHEN the stream!\n",
    "               stream = apply( func_quantum, argtuple )\n",
    "\n",
    "\n",
    "# _______________ READY-MADE GENERATORS and iterating functions:\n",
    "#\n",
    "#  N.B. -  \"Functions containing a yield statement are compiled\n",
    "#           specially as generators -- when called, they return\n",
    "#           a generator OBJECT that supports the iterator object\n",
    "#           interface [e.g. next() or .next()].\"\n",
    "#  \"next()\" is a Python built-in for iterators since v2.6.\n",
    "\n",
    "\n",
    "sip_boolean = sipstream( boolquantum,  (bestlen,)        )\n",
    "sip_trio    = sipstream( b16quantum,   (bestlen, 2)      )\n",
    "sip_nine    = sipstream( b16quantum,   (bestlen, 9)      )\n",
    "sip_hundred = sipstream( b16quantum,   (bestlen, 100)    )\n",
    "sip_real    = sipstream( realquantum,  (bestlen, NINERS ))\n",
    "sip_cent    = sipstream( realquantum,  (bestlen, 100.0)  )\n",
    "\n",
    "\n",
    "def boolean():  return sip_boolean.next()\n",
    "def trio():     return sip_trio.next() - 1\n",
    "def nine():     return sip_nine.next()\n",
    "def hundred():  return sip_hundred.next()\n",
    "def real():     return sip_real.next()\n",
    "def cent():     return sip_cent.next()\n",
    "\n",
    "\n",
    "\n",
    "def seed( length=19 ):\n",
    "    '''Create a single random integer within given length.'''\n",
    "    #  seed() turns out to be VERY useful for random integers, and nine()\n",
    "    #  as a generator greatly reduces the number of calls to the server.\n",
    "    s = ''\n",
    "    for i in range( length):\n",
    "        s += str( nine() )       \n",
    "    #  Python can represent any integer up to memory limits!\n",
    "    return int( s )\n",
    "\n",
    "\n",
    "def randint( endinteger ):\n",
    "    '''Random integer: [0, endinteger]; endinteger may be arbitrarily large!\n",
    "    '''\n",
    "    ilen = len( str(endinteger) )\n",
    "    guess = seed( ilen )\n",
    "    while guess > endinteger:\n",
    "        guess = seed( ilen )\n",
    "    return guess\n",
    "\n",
    "\n",
    "def randpick( listing, count=1, replace=True ):\n",
    "    '''Randomly pick element(s) from a list.'''\n",
    "    if replace==False  and  count > len(listing):\n",
    "        raise IndexError('Please adjust count <= length of listing.')    \n",
    "    it = listing[:] \n",
    "    #            ^need a copy, not reference, due to in-place .remove()\n",
    "    picks = []\n",
    "    for k in range( count ):\n",
    "        size = len( it )\n",
    "        lucky = randint( size-1 )\n",
    "        picks.append(  it[lucky] )\n",
    "        if not replace:\n",
    "            it.remove( it[lucky] )\n",
    "    return picks\n",
    "\n",
    "\n",
    "def shuffle( listing ):\n",
    "    '''Randomly shuffle an entire list: permutation.'''\n",
    "    #  Even for small len(x), the total number of permutations of x is \n",
    "    #  larger than the period of most PSEUDO random number generators; \n",
    "    #  this implies that most permutations of a long sequence can \n",
    "    #  never be generated. But this is not true for randquantum_authentic;\n",
    "    #  though if the listing is huge, this could take a long time to finish.\n",
    "    #\n",
    "    return randpick( listing, len(listing), replace=False )\n",
    "\n",
    "\n",
    "def gaussquantum( length, mean=0, sdev=1.0 ):\n",
    "    '''Transform random uniform to normal Gaussian distribution.\n",
    "\n",
    "    Modified from Python random module, normalvariate function.\n",
    "    Reference: Albert J. Kinderman and J.F. Monahan,\n",
    "    \"Computer generation of random variables using the ratio of \n",
    "    uniform deviates\", ACM Trans Math Software 1977, v3:3:257-260.\n",
    "\n",
    "    (Python's faster random.gauss() uses trig functions so we \n",
    "    guess that is the Box-Muller algorithm which we are avoiding\n",
    "    since it appears to constrain abs(z) to under 7.)\n",
    "\n",
    "    Ref: https://en.wikipedia.org/wiki/Normal_distribution\n",
    "    see \"Generating values\" section.\n",
    "    '''\n",
    "    NV_MAGICCONST = 1.71552776992141\n",
    "    #             = 4*exp(-0.5)/sqrt(2.0)\n",
    "    gauss = []\n",
    "    while len(gauss) < length:\n",
    "        u1 = real()\n",
    "        u2 = 1 - real()\n",
    "        z = NV_MAGICCONST*(u1-0.5)/u2\n",
    "        #   z is the KEY ratio essentially between\n",
    "        #     two random reals both from uniform distribution.\n",
    "        #     It is also the multiplier to standard deviation.\n",
    "        zz = z*z/4.0\n",
    "        #  Possible rejection next...\n",
    "        if zz <= -log(u2):\n",
    "            gauss.append( mean + (z * sdev))\n",
    "            #  Approx. acceptance rate: 73% for <= condition.\n",
    "    return gauss[:length]\n",
    "\n",
    "\n",
    "# _______________ READY-MADE GENERATOR for standard Gaussian distribution:\n",
    "\n",
    "sip_gauss   = sipstream( gaussquantum, (bestlen, 0, 1.0) )\n",
    "def gauss():    return sip_gauss.next()\n",
    "\n",
    "\n",
    "#  Interesting discussion regarding generating Gaussian distribution: \n",
    "#  http://stackoverflow.com/questions/75677/converting-a-uniform-distribution-to-a-normal-distribution\n",
    "\n",
    "\n",
    "'''\n",
    "============================================================ CONCLUSION ====== \n",
    "\n",
    "USAGE of these generators is very simple, for example:\n",
    "\n",
    "     import randquantum as rq\n",
    "     x = rq.gauss()\n",
    "     y = rq.gauss()\n",
    "\n",
    "So x and y are independently drawn from a standard \n",
    "Gaussian distribution, i.e. the normal N(0,1), \n",
    "and of course, they will be uncorrelated.\n",
    "\n",
    "Illustrating random 0 or 1:\n",
    "\n",
    "     if rq.boolean():\n",
    "          print \"yes, it's true.\"\n",
    "     else:\n",
    "          print \"nothingness.\"\n",
    "\n",
    "Such generator based functions can be invoked anywhere \n",
    "in your program without worrying about exhausting \n",
    "its list of origin. Our generators are designed to \n",
    "refresh automatically, and iterate indefinitely \n",
    "without blowing up memory space.\n",
    "\n",
    "\n",
    "FOUR UNUSUAL GENERATORS:\n",
    "rq.trio()    yields one of three equally possible: -1, 0, 1.\n",
    "rq.nine()    yields a single integer 0 through 9    inclusive.\n",
    "rq.cent()    will be real-valued between [0, 100.0] inclusive.\n",
    "rq.hundred() will be an integer between  [0, 100]   inclusive.\n",
    "\n",
    "THREE TRADITIONAL GENERATORS:\n",
    "rq.boolean() yields binary-valued: 0 or 1.\n",
    "rq.real() is real-valued [0,1] where both endpoints are included.\n",
    "rq.gauss() is drawn from the standard normal distribution N(0,1).\n",
    "\n",
    "\n",
    "FAQ:     What is the hit on performance versus pseudo random?\n",
    "Answer:  Just as fast, except those milliseconds when a tiny json file\n",
    "         is downloading. This is where the speed of your internet\n",
    "         connection matters. You could cache in the background.\n",
    "\n",
    "\n",
    "FAQ:     Why do these generators not output continuous values?\n",
    "Answer:  Well, these true random numbers originate from \n",
    "         the **quantum** world where fundamentally everything\n",
    "         is discrete.\n",
    "\n",
    "Enjoy!\n",
    "'''\n",
    "\n",
    "\n",
    "# _______________ ALTERNATE SOURCES of randomness\n",
    "#\n",
    "#  - Humboldt University, Berlin: http://qrng.physik.hu-berlin.de \n",
    "#       Uses photon arrival times. Site requires registration.\n",
    "#\n",
    "#  - http://www.random.org was cool, but now has turned commercial.\n",
    "#       Uses atmospheric noise, so it's empirical rather than \n",
    "#       theoretical in nature.\n",
    "\n",
    "\n",
    "# _______________ TESTING randomness\n",
    "#\n",
    "#  - Start at the NIST, U.S. National Institute for Standards and Technology,\n",
    "#          Computer Security Division: http://csrc.nist.gov/groups/ST/toolkit/rng\n",
    "#\n",
    "#  - Check for UPDATES at https://git.io/randomsys\n",
    "\n",
    "\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "     print \"\\n ::  THIS IS A MODULE for import -- not for direct execution! \\n\"\n",
    "     raw_input('Enter something to get out: ')\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}