m-labs · Aug 10, 2015 · Aug 10, 2015 · Aug 10, 2015
Showing with 48 additions and 6 deletions.

+1 −1 doc/manual/installing.rst

+47 −5 doc/manual/management_system.rst
diff --git a/doc/manual/installing.rst b/doc/manual/installing.rst
@@ -294,7 +294,7 @@ This should be done after either installation methods (conda or source).
 
         You should see something like this in the serial console: ::
 
-            ~/dev/misoc$ ./tools/flterm --port /dev/ttyUSB1
+            $ ./tools/flterm --port /dev/ttyUSB1
             [FLTERM] Starting...
 
             MiSoC BIOS   http://m-labs.hk

diff --git a/doc/manual/management_system.rst b/doc/manual/management_system.rst
@@ -35,16 +35,58 @@ The GUI client connects to the master and is the main way of interacting with it
 Experiment scheduling
 *********************
 
+Basics
+------
+
+To use hardware resources more efficiently, potentially compute-intensive pre-computation and analysis phases of other experiments is executed in parallel with the body of the current experiment that accesses the hardware.
+
+Experiments are divided into three phases that are programmed by the user:
+
+1. The preparation stage, that pre-fetches and pre-computes any data that necessary to run the experiment. Users may implement this stage by overloading the ``prepare`` method. It is not permitted to access hardware in this stage, as doing so may conflict with other experiments using the same devices.
+2. The running stage, that corresponds to the body of the experiment, and typically accesses hardware. Users must implement this stage and overload the ``run`` method.
+3. The analysis stage, where raw results collected in the running stage are post-processed and may lead to updates of the parameter database. This stage may be implemented by overloading the ``analyze`` method.
+
+.. note:: Only the ``run`` method implementation is mandatory; if the experiment does not fit into the pipelined scheduling model, it can leave one or both of the other methods empty (which is the default).
+
+The three phases of several experiments are then executed in a pipelined manner by the scheduler in the ARTIQ master: experiment A executes its preparation stage, then experiment A executes its running stage while experiment B executes its preparation stage, and so on.
+
+Priorities and timed runs
+-------------------------
+
+When determining what experiment to begin executing next (i.e. entering the preparation stage), the scheduling looks at the following factors, by decreasing order of precedence:
+
+1. Experiments may be scheduled with a due date. If there is one and it is not reached yet, the experiment is not eligible for preparation.
+2. The integer priority value specified by the user.
+3. The due date itself. The earlier the due date, the earlier the experiment is scheduled.
+4. The run identifier (RID), an integer that is incremented at each experiment submission. This ensures that, all other things being equal, experiments are scheduled in the same order as they are submitted.
+
+Pauses
+------
+
+In the run stage, an experiment may yield to the scheduler by calling the ``pause`` method. If there are other experiments with higher priority (e.g. a high-priority timed experiment has reached its due date), they are preemptively executed, and then the ``pause`` method returns. Otherwise, the ``pause`` method returns immediately.
+
+The experiment must place the hardware in a safe state and disconnect from the core device (typically, by using ``self.core.comm.close()``) before calling ``pause``.
+
+Accessing the ``pause`` method is done through a virtual device called ``scheduler`` that is accessible to all experiments. The scheduler virtual device is requested like regular devices using ``get_device`` or ``attr_device``.
+
+Multiple pipelines
+------------------
+
+Multiple pipelines can operate in parallel inside the same master. It is the responsibility of the user to ensure that experiments scheduled in one pipeline will never conflict with those of another pipeline over resources (e.g. same devices).
+
+Pipelines are identified by their name, and are automatically created (when an experiment is scheduled with a pipeline name that does not exist) and destroyed (when it runs empty).
+
+
 Git integration
 ***************
 
 The master may use a Git repository for the storage of experiment source code. Using Git has many advantages. For example, each result file (HDF5) contains the commit ID corresponding to the exact source code that produced it, which helps reproducibility.
 
 Even though the master also supports non-bare repositories, it is recommended to use a bare repository so that it can easily support push transactions from clients. Create it with e.g.: ::
 
-   mkdir experiments
-   cd experiments
-   git init --bare
+   $ mkdir experiments
+   $ cd experiments
+   $ git init --bare
 
 You want Git to notify the master every time the repository is pushed to (updated), so that it is rescanned for experiments and e.g. the GUI controls and the experiment list are updated.
 
@@ -55,11 +97,11 @@ Create a file named ``post-receive`` in the ``hooks`` folder (this folder has be
 
 Then set the execution permission on it: ::
 
-   chmod 755 hooks/post-receive
+   $ chmod 755 hooks/post-receive
 
 You may now run the master with the Git support enabled: ::
 
-   artiq_master -g -r /path_to/experiments
+   $ artiq_master -g -r /path_to/experiments
 
 Push commits containing experiments to the bare repository using e.g. Git over SSH, and the new experiments should automatically appear in the GUI.