pyiron_base.jobs.master.parallel module
The parallel master class is a metajob consisting of a list of jobs which are executed in parallel.
- class pyiron_base.jobs.master.parallel.GenericOutput
Bases:
OrderedDict
Generic Output just a place holder to store the output of the last child directly in the ParallelMaster.
- class pyiron_base.jobs.master.parallel.JobGenerator(master)
Bases:
object
Implements the functions to generate the parameter list, modify the individual jobs according to the parameter list and generate the new job names according to the parameter list.
Subclasses have to override
parameter_list()
to provide a list of (arbitrary) parameter objects andmodify_job()
and may overridejob_name()
to provide custom job names.The generated jobs are created as child job from the given master.
- job_name(parameter) str | tuple
Return new job name from parameter object. The next child job created will have this name. Subclasses may override this to give custom job names.
- Parameters:
parameter (type) – current parameter object drawn from
parameter_list
.- Returns:
job name for the next child job tuple: construct the job name via
_get_safe_job_name()
;allows any object that can be coerced to str inside the tuple
- Return type:
str
- property master
the parallel master job with which this generator was initialized
- Type:
- static modify_job(job, parameter)
Modify next job with the parameter object. job is already the newly created job object cloned from the template job, so this function has to return the same instance, but may (and should) modify it.
- Parameters:
job (
GenericJob
) – new job instanceparameter (type) – current parameter object drawn from
parameter_list
.
- Returns:
must be the given job
- Return type:
- next()
Iterate over the child jobs
- Returns:
new job object
- Return type:
- property parameter_list
list: parameter objects passed to
modify_job()
when the next job is requested.
- property parameter_list_cached
- class pyiron_base.jobs.master.parallel.ParallelMaster(project, job_name)
Bases:
GenericMaster
MasterJob that handles the creation and analysis of several parallel jobs (including master and continuation jobs), Examples are Murnaghan or Phonon calculations.
Subclasses must implement
collect_output()
. Additionally_job_generator
must be initialized with an instance ofJobGenerator
in the subclasses’ __init__.- Parameters:
project (ProjectHDFio) – ProjectHDFio instance which points to the HDF5 file the job is stored in
job_name (str) – name of the job, which has to be unique within the project
- .. attribute:: job_name
name of the job, which has to be unique within the project
- .. attribute:: status
- execution status of the job, can be one of the following [initialized, appended, created, submitted,
running, aborted, collect, suspended, refresh, busy, finished]
- .. attribute:: job_id
unique id to identify the job in the pyiron database
- .. attribute:: parent_id
job id of the predecessor job - the job which was executed before the current one in the current job series
- .. attribute:: master_id
- job id of the master job - a meta job which groups a series of jobs, which are executed either in parallel
or in serial.
- .. attribute:: child_ids
list of child job ids - only meta jobs have child jobs - jobs which list the meta job as their master
- .. attribute:: project
Project instance the jobs is located in
- .. attribute:: project_hdf5
ProjectHDFio instance which points to the HDF5 file the job is stored in
- .. attribute:: job_info_str
short string to describe the job by it is job_name and job ID - mainly used for logging
- .. attribute:: working_directory
working directory of the job is executed in - outside the HDF5 file
- .. attribute:: path
path to the job as a combination of absolute file system path and path within the HDF5 file.
- .. attribute:: version
Version of the hamiltonian, which is also the version of the executable unless a custom executable is used.
- .. attribute:: executable
Executable used to run the job - usually the path to an external executable.
- .. attribute:: library_activated
For job types which offer a Python library pyiron can use the python library instead of an external executable.
- .. attribute:: server
Server object to handle the execution environment for the job.
- .. attribute:: queue_id
the ID returned from the queuing system - it is most likely not the same as the job ID.
- .. attribute:: logger
logger object to monitor the external execution and internal pyiron warnings.
- .. attribute:: restart_file_list
list of files which are used to restart the calculation from these files.
- .. attribute:: exclude_nodes_hdf
list of nodes which are excluded from storing in the hdf5 file.
- .. attribute:: exclude_groups_hdf
list of groups which are excluded from storing in the hdf5 file.
- .. attribute:: job_type
- Job type object with all the available job types: [‘ExampleJob’, ‘ParallelMaster’,
‘ScriptJob’, ‘ListMaster’]
- .. attribute:: child_names
Dictionary matching the child ID to the child job name.
- .. attribute:: ref_job
Reference job template from which all jobs within the ParallelMaster are generated.
- .. attribute:: number_jobs_total
Total number of jobs
- collect_logfiles()
Collect the log files of the external executable and store the information in the HDF5 file. This method is currently not implemented for the ParallelMaster.
- collect_output()
Collect the output files of the external executable and store the information in the HDF5 file. This method has to be implemented in the individual meta jobs derived from the ParallelMaster.
- convergence_check() bool
Check if and all child jobs of the calculation are converged. May need be extended in the base classes depending on the specific application
- Returns:
If the calculation is converged
- Return type:
(bool)
- copy()
Copy the GenericJob object which links to the job and its HDF5 file
- Returns:
New object pointing to the same job
- Return type:
- create_child_job(job_name)
Internal helper function to create the next child job from the reference job template - usually this is called as part of the create_jobs() function.
- Parameters:
job_name (str) – name of the next job
- Returns:
next job
- Return type:
- interactive_ref_job_initialize()
To execute the reference job in interactive mode it is necessary to initialize it.
- is_finished()
Check if the ParallelMaster job is finished - by checking the job status and the submission status.
- Returns:
[True/False]
- Return type:
bool
- iter_jobs(convert_to_object=True)
Iterate over the jobs within the ListMaster
- Parameters:
convert_to_object (bool) – load the full GenericJob object (default) or just the HDF5 / JobCore object
- Returns:
Yield of GenericJob or JobCore
- Return type:
yield
- property number_jobs_total
Get number of total jobs
- Returns:
number of total jobs
- Return type:
int
- output_to_pandas(sort_by=None, h5_path='output')
Convert output of all child jobs to a pandas Dataframe object.
- Parameters:
sort_by (str) – sort the output using pandas.DataFrame.sort_values(by=sort_by)
h5_path (str) – select child output to include - default=’output’
- Returns:
output as dataframe
- Return type:
pandas.Dataframe
- property ref_job
Get the reference job template from which all jobs within the ParallelMaster are generated.
- Returns:
reference job
- Return type:
- refresh_submission_status()
Refresh the submission status - if a job ID job_id is set then the submission status is loaded from the database.
- reset_job_id(job_id=None)
Reset the job id sets the job_id to None as well as all connected modules like JobStatus and SubmissionStatus.
- run_if_interactive()
For jobs which executables are available as Python library, those can also be executed with a library call instead of calling an external executable. This is usually faster than a single core python job.
- run_if_refresh()
Internal helper function the run if refresh function is called when the job status is ‘refresh’. If the job was suspended previously, the job is going to be started again, to be continued.
- run_static()
The run_static function is executed within the GenericJob class and depending on the run_mode of the Parallelmaster and its child jobs a more specific run function is selected.
- save()
Save the object, by writing the content to the HDF5 file and storing an entry in the database.
- Returns:
Job ID stored in the database
- Return type:
(int)
- show_hdf()
Display the output of the child jobs in a human readable print out