utils

Non-numerical utilities (notably for parallel computation).

Overview

This module gathers several non-numerical utilities. The only one of direct interest to the user is the multiplexer function, which we now describe briefly.

Say we have some function f, which takes only keyword arguments:

def f(x=0, y=0, z=0):
    return x + y + z**2

We wish to evaluate f repetitively for a range of x, y and/or z values. To do so, we may use function multiplexer as follows:

results = multiplexer(f=f, x=3, y=[2, 4, 6], z=[3, 5])

which returns a list of 3*2 dictionaries of the form:

[ {'x':3, 'y':2, 'z':3, 'out':14},  # 14=f(3, 2, 3)
  {'x':3, 'y':2, 'z':5, 'out':30},
  {'x':3, 'y':4, 'z':3, 'out':16},
   ... ]

In other words, multiplexer computes the Cartesian product of the inputs.

For each argument, you may use a dictionary instead of a list:

results = multiplexer(f=f, z={'good': 3, 'bad': 5})

In that case, the values of the dictionaries are used in the same way as above, but the output reports the corresponding keys, i.e.:

[ {'z': 'good', 'out': 12},  # f(0, 0, 3)
  {'z': 'bad', 'out': 28}    # f(0, 0, 5)
]

This is useful when f takes as arguments complex objects that you would like to replace by more legible labels; e.g. option ` model` of class SMC.

multiplexer also accepts three extra keyword arguments (whose name may not therefore be used as keyword arguments for function f):

  • nprocs: if >0, number of CPU cores to use in parallel; if <=0, number of cores not to use; in particular, nprocs=0 means all CPU cores must be used.
  • nruns (default=1): evaluate f nruns time for each combination of arguments; an entry run (ranging from 0 to nruns-1) is added to the output dictionaries. This is mostly useful when the output of f is random. * seeding (default: True if nruns>1, False otherwise): if True, seeds the pseudo-random generator before each call of function f with a different seed. See second warning below.

Warning

Option nprocs rely on the standard library multiprocessing, whose performance and behaviour seems to be OS-dependent. In particular, it may not work well on Windows.

Warning

Library multiprocessing generates identical workers, up to the state of the random generator. Thus, as soon as more than one core is used, we strongly recommend to set option seeding above to True.

See also

multiSMC

Module summary

multiplexer Evaluate a function for different parameters, optionally in parallel.