opal.parser package

opal.parser.FieldParser module

class opal.parser.FieldParser.FieldParser[source]

Bases: opal.parser.BaseParser.BaseParser

Read OPAL field data generated with compile flag DBG_SCALARFIELD enabled.

_columns

the keys are the column names and the values their units

Type

dict

_df

the data

Type

pandas.core.frame.DataFrame

_dim

list of number of grid points per dimension

Type

list

_stride

list of stride per dimension

Type

list

__init__()[source]

Constructor

_check_header(filename)[source]
_parse_header(filename)[source]
clear()[source]

Reset attributes

property dataframe

Get all data.

Returns

all the data

Return type

pandas.core.frame.DataFrame

property dimension

returns: the number of grid points per dimension :rtype: list

property field

returns: the field data on the grid :rtype: numpy.ndarray

get_unit_dictionary()[source]

Get the dictionary of units.

Returns

column names (keys) and their units (values)

Return type

dict

property image

returns: the image potential (if available) :rtype: numpy.array

property indices

returns: the grid point indices :rtype: numpy.ndarray

is_scalar()[source]
Returns

True if a scalar field

Return type

bool

parse(filename)[source]

Parse a file.

Parameters

filename (str) – the full name of a file to parse

property positions

returns: the positions at the grid points :rtype: numpy.ndarray

property stride

returns: the stride per dimension :rtype: list

opal.parser.H5Error module

exception opal.parser.H5Error.H5AttributeError(msg)[source]

Bases: opal.parser.H5Error.H5Error

Exception raised for errors concerning attributes.

__init__(msg)[source]

Initialize self. See help(type(self)) for accurate signature.

exception opal.parser.H5Error.H5DatasetError(msg)[source]

Bases: opal.parser.H5Error.H5Error

Exception raised for errors concerning datasets.

__init__(msg)[source]

Initialize self. See help(type(self)) for accurate signature.

exception opal.parser.H5Error.H5Error(msg)[source]

Bases: Exception

Base class for H5 exceptions. (see also https://docs.python.org/2/tutorial/errors.html)

__msg

explanation of the error

Type

str

__init__(msg)[source]

Initialize self. See help(type(self)) for accurate signature.

exception opal.parser.H5Error.H5OverflowError(msg)[source]

Bases: opal.parser.H5Error.H5Error

Exception raised for errors in accessing data.

__init__(msg)[source]

Initialize self. See help(type(self)) for accurate signature.

exception opal.parser.H5Error.H5ParseError(msg)[source]

Bases: opal.parser.H5Error.H5Error

Exception raised for errors in parsing.

__init__(msg)[source]

Initialize self. See help(type(self)) for accurate signature.

opal.parser.H5Parser module

class opal.parser.H5Parser.H5Parser[source]

Bases: opal.parser.BaseParser.BaseParser

__init__()[source]

Constructor.

_check_step(step)[source]
clear()[source]

Clear data.

close()[source]

Closes h5 file.

getGlobalAttribute(attrName)[source]

Obtain value of global attribute.

Parameters

attrName (str) – Name of global attribute

Returns

Value as string.

Return type

str

getGlobalAttributes()[source]

All attributes of root.

Returns

A list of strings of all root attributes

Return type

list

getNSteps()[source]

Number of steps in file.

Returns

The number of steps

Return type

int

getStepAttribute(attrName, step=0)[source]

Obtain the attribute data of a step.

Parameters

  • attrName (str) – Name of attribute

  • step (int, optional) – Steps of file

getStepAttributes(step=0)[source]

Attributes names of a step.

Parameters

step (int, optional) – Steps of file

Returns

All attribute names of a step

Return type

list [str]

getStepDataset(dsetName, step=0)[source]

Obtain the data of a step.

Parameters

  • dsetName (str) – Name of dataset

  • step (int, optional) – Steps of file

Returns

Array of a dataset of a step (check with getStepDatasets())

Return type

numpy.ndarray, (N,)

getStepDatasets(step=0)[source]

All step dataset names.

Parameters

step (int, optional) – Steps of file

Returns

All attribute names of a step

Return type

list [str]

parse(fname, info=False)[source]

Read a H5 file generated by OPAL

Parameters

  • fname (str) – H5 filename

  • info (bool, optional) – Print H5 file summary

opal.parser.HistogramParser module

class opal.parser.HistogramParser.HistogramParser[source]

Bases: opal.parser.BaseParser.BaseParser

Parses OPAL *.hist files

Parses *.hist files of OPAL that are written for probe elements and *.hist files of measurements.

The first line in both files is a comment line starting with ‘#’. The measurement header line should be:

# Histogram bin counts (radius ['unit'], intensity ['unit'])

where a simulation header is:

# Histogram bin counts (min, max, nbins, binsize) 'min' 'unit' 'max' 'unit' '#bins' 'binsize' 'unit'
_info

Dictionary with histogram info:

‘min’

List of minimum value and unit

‘max’

List of maximum value and unit

‘nbins’

List of number of bins and unit

‘binsize’

List of size of bin and unit

‘bincount’

List of bin counts and unit

Type

dict

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

clear()[source]

Clear data.

getDataOfVariable(var)[source]
getUnitOfVariable(var)[source]
getVariables()[source]
isVariable(var)[source]
parse(filename)[source]
property size

opal.parser.LatticeParser module

class opal.parser.LatticeParser.LatticeParser[source]

Bases: object

__init__()[source]

Constructor.

plot(lfile, fig, ax1, ax2)[source]

opal.parser.LossParser module

class opal.parser.LossParser.LossParser[source]

Bases: opal.parser.BaseParser.BaseParser

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

clear()[source]

Clear data.

getDataOfVariable(var)[source]
getUnitOfVariable(var)[source]
getVariableNames()[source]
isVariable(var)[source]
parse(filename)[source]

Parser method for loss files

Examples

1. Clean

(OPAL version <= 2.4) line = “# Element STQ1 x (m), y (m), z (m), px ( ), py ( ), pz ( ), id, turn, time (ns)” (OPAL version >= 2.6) line = “# x (m), y (m), z (m), px ( ), py ( ), pz ( ), id, turn, time (s)”

gets

line = ” x (m), y (m), z (m), px ( ), py ( ), pz ( ), id, turn, time (s)”

2. no commas

line = ” x (m), y (m), z (m), px ( ), py ( ), pz ( ), id, turn, time (s)”

gets

line = ” x (m) y (m) z (m) px ( ) py ( ) pz ( ) id turn time (s)”

property size

opal.parser.OptimizerParser module

class opal.parser.OptimizerParser.OptimizerParser[source]

Bases: opal.parser.BaseParser.BaseParser

Parsing of JSON files generated by the OPAL optimizer

__ids

List of all individual IDs of a generation

Type

list

__dvar_values

List of all design variables values for each ID (order corresponds to __ids)

Type

numpy.matrix

__obj_values

List of all objective values for each ID (order corresponds to __ids)

Type

numpy.matrix

__obj_names

Specifies columns in self.__obj_values

Type

list

__dvar_names

Specifies columns in self.__dvar_values

Type

list

__pareto
Type

str

Notes

Supports following JSON files:

{
    "name": "opt-pilot",
    "dvar-bounds": {
        "benergy": [ 0.071, 0.072 ],
        "phiinit": [ 106, 114 ],
        "prinit": [ -0.02, -0.01 ],
        "rinit": [ 2000, 2060 ]
    }
    ,
    "constraints": [
        "dpeak1 > 0.0",
        "dpeak2 > 0.0"
    ]
    ,
    "solutions": [
    {
        "ID": 0,
        "obj":
        {
                "dpeak1": 10.37,
                "dpeak2": 5.43,
                "dpeak3_5": 14.1095
        }
        ,
        "dvar":
        {
                "benergy": 0.0718672,
                "phiinit": 113.653,
                "prinit": -0.0191565,
                "rinit": 2018.26
        }
    }
    ,
    {
        "ID": 18,
        "obj":
        {
                "dpeak1": 5.95,
                "dpeak2": 4.75,
                "dpeak3_5": 17.6668
        }
        ,
        "dvar":
        {
                "benergy": 0.0715704,
                "phiinit": 110.517,
                "prinit": -0.0122177,
                "rinit": 2035.32
        }
    }
    ,
    {
        "ID": 2,
        "obj":
        {
                "dpeak1": 1.84,
                "dpeak2": 6.04,
                "dpeak3_5": 19.2461
        }
        ,
        "dvar":
        {
                "benergy": 0.0711942,
                "phiinit": 107.097,
                "prinit": -0.0198867,
                "rinit": 2034.75
        }
    }
    ]
}

{
    "name": "opt-pilot",
     "OPAL version": "2.0.0",
     "git revision": "1849b7e5130657e8be50d524de0f6c50134f330a",
     "dvar-bounds": {
         "MX": "[ 16, 32 ]",
         "nstep": "[ 10, 40 ]"
     },
     "constraints": [
         "statVariableAt('rms_x',0.0) > 0",
         "statVariableAt('rms_x',0.0) > 0.001"
     ],
     "population": {
         "0": {
             "obj": {
                 "dpeak1": 10.37,
                 "dpeak2": 5.43,
                 "dpeak3_5": 14.1095
             },
             "dvar": {
                 "benergy": 0.0718672,
                 "phiinit": 113.653,
                 "prinit": -0.0191565,
                 "rinit": 2018.26
             }
         },
         "18": {
             "obj": {
                 "dpeak1": 5.95,
                 "dpeak2": 4.75,
                 "dpeak3_5": 17.6668
             },
             "dvar": {
                 "benergy": 0.0715704,
                 "phiinit": 110.517,
                 "prinit": -0.0122177,
                 "rinit": 2035.32
             }
         },
         "2": {
             "obj": {
                 "dpeak1": 1.84,
                 "dpeak2": 6.04,
                 "dpeak3_5": 19.2461
             },
             "dvar": {
                 "benergy": 0.0711942,
                 "phiinit": 107.097,
                 "prinit": -0.0198867,
                 "rinit": 2034.75
             }
         }
     }
 }

Examples

>>> import OptimizerParser as optreader
>>> import sys
>>> try:

  1. Find all .json files of a directory, e.g. “./”

>>> optjson = optreader.OptimizerParser()
>>> optjson.parse("./")

  1. Read in a generation file, e.g. 1

>>> optjson.readGeneration(1)

  1. Get all design variables

>>> dvars = optjson.getDesignVariables()
>>> print ( "Design variables: ", dvars )

  1. Get design variable bounds

>>> bounds = optjson.getBounds()
>>> print ( "Bounds: ", bounds )

4a. Get only bounds of a specific design variable

>>> bound = optjson.getBounds(dvars[2])
>>> lower = bound[0]
>>> upper = bound[1]
>>> print ( "dvar: " + dvars[1] + "
>>>     " lower bound: " + str(lower) + "
>>>     " upper bound: " + str(upper))

  1. Get all objectives

>>> objs = optjson.getObjectives()
>>> print ( "Objectives: ", objs )

  1. Get all constraints

>>> constr = optjson.getConstraints()
>>> print ("Constraints: " + constr[0] + " " + constr[1])

  1. Get an individual

>>> print ( optjson.getIndividual(0) )

  1. Get an individual with specific ID

>>> print ( optjson.getIndividualWithID(18) )

raise error

>>> except:
    >>> print ( sys.exc_info()[1] )
__actual_parse(filename)

Parse an optimizer JSON file

Parameters

filename (str) – JSON file to be loaded

__clear_container()

Clear old attribute data

__hasLoaded()

Throw error if no generation or pareto front file loaded.

__init__()[source]

Constructor.

__parse_version_2_0_0(data)

Read in data

Parameters

data (str) – A generation file

Notes

Storage of columns: dvars, objs, ID

__parse_version_2_1_0(data)

Read in data

Parameters

data (str) – A generation file

__reset_attributes()

Reset attributes to default values.

check_file(filename)[source]

Check if a file is really an optimizer output

Parameters

filename (str) – JSON file to be loaded

Returns

True if an optimizer file, otherwise False

Return type

bool

clear()[source]

Clear data.

getAllInput()[source]

Obtain all design variable input.

Returns

A ndarray where each column corresponds to the values of a design variable

Return type

numpy.ndarray

getAllOutput()[source]

Obtain all objective output.

Returns

A ndarray where each column corresponds to the values of an objective

Return type

numpy.ndarray

getBounds(dvar='')[source]

Obtain the design variable bounds

Parameters

dvar (str, optional) – Design variable or ‘’

Returns

If dvar == ‘’ it returns the whole dictionary of design variables and their bounds otherwise it returns the bounds of the given design variable

Return type

dict or array

Notes

Raises an exception if design variable unknown

getConstraints()[source]

Obtain the constraints of the simulation. They are formulas written as strings

Returns

A list of strings containing the constraints

Return type

list [str]

getDesignVariables()[source]

Obtain names of all design variables

Returns

A list containing all design variable names

Return type

list

getIDs()[source]

Obtain all individual IDs.

Returns

A list of IDs

Return type

list [int]

getIndexOfID(ID)[source]

Get Index of ID

Returns

The index of the individual with certain ID

Return type

int

getIndividual(ind)[source]

Get individual in a list of [dvars, objs, ID]

Parameters

ind (int) – The individual number (not the ID)

Returns

A list containing [design variables, objectives, ID]

Return type

list

Notes

Raises an index error if individual number is out of bounds

getIndividualWithID(ID)[source]

Get invdividual with ID

Parameters

ID (int) – ID for which we want the data

Returns

A list containing [design variables, objectives, ID]

Return type

list

Notes

Raises a runtime error if there’s no individual with given ID

getNumOfGenerations()[source]
getObjectives()[source]

Obtain names of all objectives

Returns

A list containing all design variable names

Return type

list

property num_optimizers
parse(directory)[source]

Collect information about optimizer JSON files.

Parameters

directory (str) – Where the .json files are

Notes

Raises a runtime error if

  • there are several several different basenames

  • no json file is found

readGeneration(gen, opt=0, pareto=False)[source]

Parse a generation file

Parameters

  • gen (int) – The generation number

  • opt (int, optional) – The optimizer number (default: 0)

  • pareto (str, optional) – Read a pareto file (default: False)

Notes

Raises a runtime error if generation file doesn’t exist

opal.parser.PeakParser module

class opal.parser.PeakParser.PeakParser[source]

Bases: opal.parser.BaseParser.BaseParser

Read measurement file of the form:

# Peak Radii (mm), 18 June 2008, 200 uA (RRI2-65)
2103.8
2122.4
2140.4
2159.8
2180.8
2201.8
2224.6
2247.8
2270.0
2287.8
2302.6
2315.6
2331.4
2356.0
2379.6
2403.6
2424.6

It is also able to readd simulation files of the same form.

_peaks

Dictionary with peak file info =

‘type’

Simulation or measurement

‘name’

List of all supported variables

‘dataset’

List of radii and intensity

‘unit’

List of units

Type

dict

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

clear()[source]

Clear data.

getDataOfVariable(var)[source]
getType()[source]
getUnitOfVariable(var)[source]
getVariableName()[source]
isVariable(var)[source]
parse(filename)[source]

opal.parser.SDDSParser module

class opal.parser.SDDSParser.SDDSParser[source]

Bases: opal.parser.BaseParser.BaseParser

__removeNumber(s)
_checkVersion(filename)[source]
_column(f)[source]
_data(f)[source]
_description(f)[source]
_get_type(dtype)[source]
_hasVariable(varname)[source]
_parameter(f)[source]
_parseHeader1_6(filename)[source]
_parseHeader1_9(filename)[source]
_version(v)[source]
clear()[source]

Clear data.

collectStatFileData(baseFN, root, yNames)[source]

FIXME: This function shouldn’t be part of the parser.

Assumes runOPAL structure: optLinac_40nC_IBF=485.9269768907996_IM … where baseFN == optLinac_40nC in the example above.

This function finds all stat files that are one level below root. An exclude list can be specified.

Two vectors are returned: x with the design variables names and values (IBF=485.9269768907996 … from above) and the last value(s) of stat file data spcified via yNames.

Example: x = [] y = [] baseFN = ‘optLinac_40nC’ exclList = [‘tmpl’,’extrData.py’] root = “.” yNames = [‘s’,’energy’] p = SDDSParser() (x,y) = p.makeData(baseFN, exclList, root, yNames)

property dataframe
getDataOfVariable(varname)[source]
getDescriptionOfVariable(varname)[source]
getUnitOfVariable(varname)[source]
getVariables()[source]
parse(filename)[source]
property size

opal.parser.TimingParser module

class opal.parser.TimingParser.TimingParser[source]

Bases: opal.parser.BaseParser.BaseParser

Read and write an Ippl timing file.

_data

List of dictionaries

Type

list

Notes

It stores the data in a list of dictionary where the main timing dictionary is:

main_dict = {'cpu tot':     [],
             'wall tot':    [],
             'what':        [],
             'cores':       []}

and the specialized timings are in stored in a dictionary with the structure:

special_dict = {'what':     [],
                'cpu max':  [],
                'wall max': [],
                'cpu min':  [],
                'wall min': [],
                'cpu avg':  [],
                'wall avg': []}

The list of dictionaries looks then as follows:

self._data = [main_dict,
              special_dict_1,
              ...,
              special_dict_N]

Examples

>>> TimingParser as timing
>>> time = timing.TimingParser()
>>> time.read_ippl_timing("/path/to/IpplTiming.dat")
>>> data = time.getTiming()
>>> print ( data )
>>> time.pie_plot('cpu avg'):
__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

_exportAscii(pathname, data)[source]

Write a human readable file

Parameters

  • pathname (str) – Path + filename of written file

  • data (list [dict]) – Timing data

Notes

If pathname has no extension the string “.dat” is appended

_exportPickle(pathname, data)[source]

Write a binary pickle file

Parameters

  • pathname (str) – Path + filename of written file

  • data (list [dict]) – Timing data

Notes

If pathname has no extension the string “.pkl” is appended

_init_data_structure()[source]

This is the way the timing is stored

Returns

  • main_dict (dict) – The dictionary for the main timer

  • special_dict (dict) – The dictionary for all other timers

_load_pkl(pkl_file)[source]

Pickle file loading function

Parameters

pkl_file (str) – Pickle timing file to load

_order(line, i)[source]

Find the order of the tags, i.e. ‘cpu min’, etc. and fill dictionary.

Parameters

  • line (str) – File line

  • i (int) – Start index

Returns

A dictionary giving tag as key and occurrence as number.

Return type

dict

clear()[source]

Clear data.

getProblemSize()[source]
Returns

All problem specification in a dictionary

Return type

dict

Notes

It is not checked if the container is empty.

getTiming()[source]
Returns

The timing data

Return type

list

Notes

It is not checked if the container is empty.

parse(filename)[source]
property properties
read(pathname, info=False)[source]
Parameters

  • pathname (str) – Path + filename of pickle file

  • info (bool, optional) – Print data when reading

read_ippl_timing(f)[source]

Read in an Ippl timing file.

File created by:

std::string filename = "myTiming.dat";
Ippl:print(filename, problemSize);

The problem size is optional.

Parameters

f (str) – Pathname (i.e. path + filename)

read_output_file(f)[source]

Read in the timing results from an OPAL output file.

Parameters

f (str) – The pathname (i.e. path + filename)

Notes

Following format assumed:

Timings{0}> -----------------------------------------------------------------
Timings{0}>      Timing results for 32 nodes:
Timings{0}> -----------------------------------------------------------------
Timings{0}> mainTimer........... Wall tot =    326.192, CPU tot =     325.76
Timings{0}>
Timings{0}> my awesome timer.... Wall max =          0, CPU max =          0
Timings{0}>                      Wall avg =          0, CPU avg =          0
Timings{0}>                      Wall min =          0, CPU min =          0
Timings{0}>
Timings{0}> super timer......... Wall max =    14.6091, CPU max =      14.52
Timings{0}>                      Wall avg =    3.34291, CPU avg =    3.31844
Timings{0}>                      Wall min =   0.007039, CPU min =          0
Timings{0}>
Timings{0}> best timer.......... Wall max =    33.4165, CPU max =      32.93
Timings{0}>                      Wall avg =    23.0727, CPU avg =    22.8328
Timings{0}>                      Wall min =     19.989, CPU min =      19.67
Timings{0}>
Timings{0}> -----------------------------------------------------------------
write(pathname, form='PICKLE', data=None)[source]

Export a timing data in a specific format

Parameters

  • pathname (str) – Path + name of the written file

  • data (list [dict], optional) – Timing data

  • form (str, optional) – Which format to write

Notes

Throws an exception if the format is unknown or not available

opal.parser.TrackOrbitParser module

class opal.parser.TrackOrbitParser.TrackOrbitParser[source]

Bases: opal.parser.BaseParser.BaseParser

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

clear()[source]

Clear data.

property description
getDataOfVariable(var)[source]
getUnitOfVariable(var)[source]
getVariableNames()[source]
isVariable(var)[source]
parse(filename)[source]
property size

opal.parser.sampler module

class opal.parser.sampler.SamplerParser[source]

Bases: opal.parser.BaseParser.BaseParser

Parses *.json files of OPAL that are written by the SAMPLE command.

Notes:

Supports following JSON formats:

{
    "name": "sampler",
    "dvar-bounds": {
        "MX": "[ 16, 32 ]",
        "nstep": "[ 10, 40 ]"
    },
    "samples": [
        {
            "ID": "0",
            "dvar": {
                "MX": "20",
                "nstep": "33"
            }
        },
        {
            "ID": "1",
            "dvar": {
                "MX": "21",
                "nstep": "29"
            }
        }
    ]
}

{
    "samples": {
        "0": {
            "dvar": {
                "MX": "16",
                "nstep": "10"
            },
            "obj": {
                "o1": 22.2,
                "o2": 22.1
            }
        },
        "1": {
            "dvar": {
                "MX": "19",
                "nstep": "11"
            },
            "obj": {
                "o1": 21.2,
                "o2": 23.1
            }
        },
        "2": {
            "dvar": {
                "MX": "22",
                "nstep": "12"
            },
            "obj": {
                "o1": 12.2,
                "o2": 32.1
            }
        }
    },
    "name": "sampler",
    "OPAL version": "2.0.0",
    "git revision": "1849b7e5130657e8be50d524de0f6c50134f330a",
    "dvar-bounds": {
        "MX": "[ 16, 32 ]",
        "nstep": "[ 10, 40 ]"
    }
}
__id

(= ‘sampler’) used to identify the file to be a SAMPLE output

Type

str

__tag

(= ‘name’) id tag for file identification

Type

str

__begin

Start individual id

Type

int

__end

End individual id

Type

int

__dvars

The design variables of each individual

Type

list

__dvar_bounds

All design variable bounds

Type

dict

__objs

The objectives of each individual

Type

list

__fix_bound_type()

Fixes type of DVAR bounds. In the JSON file the bounds are in a string, e.g. ‘[0, 1]’.

We need to change to list of floats.

__init__()[source]

Initialize self. See help(type(self)) for accurate signature.

__parse_version_2_0_0(data)
__parse_version_2_1_0(data)
property begin

returns: Lowest individual ID :rtype: int

property bounds

Obtain design variable upper and lower bounds

Returns

A dictionary of design variable names (key) and their bounds

Return type

dict

clear()[source]

Clear data.

property design_variables

Obtain names of design variables.

Returns

A list of strings

Return type

list [str]

property end

returns: Highest individual ID :rtype: int

getIndividual(ind)[source]

Obtain input values of an individual

Parameters

ind (int) – Individual number

Returns

A dictionary with design variable names (keys) and their input value.

Return type

dict

getObjectives(ind)[source]

Obtain output values of an individual

Parameters

ind (int) – Individual number

Returns

A dictionary with design variable names (keys) and their input value.

Return type

dict

property num_samples
property objectives

Obtain names of objectives.

Returns

A list of strings

Return type

list [str]

parse(filename)[source]

Load the JSON file.

Parameters

filename (str) – JSON file to be loaded