Quickstart#

partialwrap: a small Python library providing wrappers for external executables to be used easily with Python’s functools.partial().

About partialwrap#

partialwrap is a Python library providing easy wrapper functions to use with Python’s functools.partial(). Partial’s ingenious mechanism allows to use even very complex functions with many arguments and keyword arguments with routines that need functions in the simple form func(x). This includes Python’s map() function, the minimizers in scipy.optimize, and plenty of third-party modules such as emcee or pyeee. partialwrap allows to use any external executable as well as any Python function with arbitrary arguments and keywords to be used with functools.partial(). It also allows to use the executables or functions with easy parallelization of code with, for example, the parallel map function of the multiprocessing module or via the Message Passing Interface (MPI) mpi4py.

Quick usage guide#

partialwrap provides two wrapper functions to work with external executables: exe_wrapper() and exe_mask_wrapper().

partialwrap writes the input arguments to the wrapper functions into files that can be read by the external executable. The executable should write its result to a file that will then be read by partialwrap in return. This means partialwrap needs to have a function parameterwriter that writes the parameter file parameterfile in the form needed by the executable exe. It then also needs to have a function outputreader for reading the output file outputfile of the external executable, perhaps calculating an objective function value.

Take the Rastrigin function (https://en.wikipedia.org/wiki/Rastrigin_function), which is a popular function for performance testing of optimization algorithms: \(y = a \cdot n + \sum_i^n (x_i^2 - a \cos(b \cdot x_i))\). It has a global minimum of \(0\) at all \(x_i = 0\). \(a\) influences mainly the depth of the (local and global) minima, whereas \(b\) influences mainly the size of the minima. A common form uses \(a = 10\) and \(b = 2 \pi\). The parameters \(x_i\) should then be in the interval \([-5.12, 5.12]\).

Consider for simplicity an external Python program (e.g. rastrigin1.py) that calculates the Rastrigin function with \(a = 10\) and \(b = 2 \pi\), reading in an arbitrary number of parameters \(x_i\) from a parameterfile = params.txt and writing its output into an outputfile = out.txt:

# File: rastrigin1.py
import numpy as np
from partialwrap import standard_parameter_reader

# Rastrigin function a=10, b=2*pi
def rastrigin1(x):
    return 10. * len(x) + np.sum(x**2 - 10. * np.cos(2. * np.pi * x))

# read parameters
x = standard_parameter_reader('params.txt')

# calc function
y = rastrigin1(x)

# write output file
with open('out.txt', 'w') as ff:
    print(y, file=ff)

This program can be called on the command line (if params.txt is present) with:

python3 rastrigin1.py

The external program calculating the Rastrigin function could, of course, also be written in any compiled language such as C or Fortran. See the User Guide for details. The external program, here the Python version, can be used with Python’s functools.partial() and the wrapper function exe_wrapper():

import scipy.optimize as opt
from functools import partial
from partialwrap import exe_wrapper
from partialwrap import standard_parameter_writer, standard_output_reader

rastrigin_exe   = ['python3', 'rastrigin1.py']
parameterfile   = 'params.txt'
parameterwriter = standard_parameter_writer
outputfile      = 'out.txt'
outputreader    = standard_output_reader
rastrigin_wrap  = partial(exe_wrapper, rastrigin_exe,
                          parameterfile, parameterwriter,
                          outputfile, outputreader, {})

x0  = [0.1, 0.2, 0.3]
res = opt.minimize(rastrigin_wrap, x0, method='BFGS')

The scipy.optimize function minimize() passes its sampled parameters to exe_wrapper, which writes it to the file parameterfile = ‘params.txt’. It then calls rastrigin_exe = ‘python3 rastrigin1.py’ and reads its outputfile = ‘out.txt’. standard_parameter_reader() and standard_parameter_writer() are convenience functions that read and write one parameter per line in a file without a header. The empty dictionary at the end is explained in the User Guide.

More elaborate input/output of the external program can simply be done by replacing standard_parameter_reader() and standard_parameter_writer() with appropriate functions, while the rest stays pretty much the same.

Installation#

The easiest way to install is via pip:

pip install partialwrap

or via conda:

conda install -c conda-forge partialwrap

Requirements#

numpy

License#

partialwrap is distributed under the MIT License. See the LICENSE file for details.

The project structure is based on a template provided by Sebastian Müller.

Quickstart

Contents