pyiron_base.jobs.job.path.JobPath

pyiron_base.jobs.job.path.JobPath#

class pyiron_base.jobs.job.path.JobPath(job_path: str)[source]#

Bases: JobCore

The JobPath class is derived from the JobCore and is used as a lean version of the GenericJob class. Instead of loading the full pyiron object the JobPath class only provides access to the HDF5 file, which should be enough for most analysis.

.. attribute:: job_name: name of the job, which has to be unique within the project

.. attribute:: status

execution status of the job, can be one of the following [initialized, appended, created, submitted, running,: aborted, collect, suspended, refresh, busy, finished]

.. attribute:: job_id: unique id to identify the job in the pyiron database

.. attribute:: parent_id: job id of the predecessor job - the job which was executed before the current one in the current job series

.. attribute:: master_id: job id of the master job - a meta job which groups a series of jobs, which are executed either in parallel or in serial.

.. attribute:: child_ids: list of child job ids - only meta jobs have child jobs - jobs which list the meta job as their master

.. attribute:: project: Project instance the jobs is located in

.. attribute:: project_hdf5: ProjectHDFio instance which points to the HDF5 file the job is stored in

.. attribute:: job_info_str: short string to describe the job by it is job_name and job ID - mainly used for logging

.. attribute:: working_directory: working directory of the job is executed in - outside the HDF5 file

.. attribute:: path: path to the job as a combination of absolute file system path and path within the HDF5 file.

.. attribute:: is_root: boolean if the HDF5 object is located at the root level of the HDF5 file

.. attribute:: is_open: boolean if the HDF5 file is currently opened - if an active file handler exists

.. attribute:: is_empty: boolean if the HDF5 file is empty

.. attribute:: base_name: name of the HDF5 file but without any file extension

.. attribute:: file_path: directory where the HDF5 file is located

.. attribute:: h5_path: path inside the HDF5 file - also stored as absolute path

__init__(job_path: str)[source]#

Load a job from the given path.

Parameters:: job_path (str) – path to the job, must be of the form /path/to/file.h5/job_name

Methods

`__init__`(job_path)	Load a job from the given path.
`check_if_job_exists`([job_name, project])	Check if a job already exists in an specific project.
`close`()	Close the current HDF5 path and return to the path before the last open
`compress`([files_to_compress, files_to_remove])	Compress the output files of a job object.
`copy`()	Copy the JobCore object which links to the HDF5 file
`copy_to`(project[, new_job_name, input_only, ...])	Copy the content of the job including the HDF5 file to a new location
`create_group`(name)	Create an HDF5 group - similar to a folder in the filesystem - the HDF5 groups allow the users to structure their data.
`decompress`()	Decompress the output files of a compressed job object.
`from_db_entry`(db_entry)	Load a job path from a database entry.
`from_hdf`([hdf, group_name])	Restore object from hdf5 format - The function has to be implemented by the derived classes - usually the GenericJob class
`from_job_id`(db, job_id)	Load a job path from a database connection and the job id.
`get`(name[, default])	Internal wrapper function for __getitem__() - self[name]
`get_from_table`(path, name)	Get a specific value from a pandas.Dataframe
`get_job_id`([job_specifier])	get the job_id for job named job_name in the local project path from database
`groups`()	Filter HDF5 file by groups
`inspect`(job_specifier)	Inspect an existing pyiron object - most commonly a job - from the database
`is_compressed`()	Check if the job is already compressed or not.
`is_master_id`(job_id)	Check if the job ID job_id is the master ID for any child job
`is_self_archived`()	Check if the HDF5 file of the Job is compressed as tar-archive
`items`()	List all keys and values as items of all groups and nodes of the HDF5 file
`keys`()	List all groups and nodes of the HDF5 file - where groups are equivalent to directories and nodes to files.
`list_all`()	Returns dictionary of :method:`.list_groups()` and :method:`.list_nodes()`.
`list_childs`()	List child jobs as JobPath objects - not loading the full GenericJob objects for each child
`list_dirs`()	equivalent to os.listdirs (consider groups as equivalent to dirs)
`list_files`()	List files inside the working directory
`list_groups`()	Return a list of names of all nested groups.
`list_nodes`()	Return a list of names of all nested nodes.
`listdirs`()	equivalent to os.listdirs (consider groups as equivalent to dirs)
`load`(job_specifier[, convert_to_object])	Load an existing pyiron object - most commonly a job - from the database
`move_to`(project)	Move the content of the job including the HDF5 file to a new location
`nodes`()	Filter HDF5 file by nodes
`open`(h5_rel_path)	Create an HDF5 group and enter this specific group.
`put`(key, value)	Store data inside the HDF5 file
`relocate_hdf5`([h5_path])	Relocate the hdf file.
`remove`([_protect_childs])	Remove the job - this removes the HDF5 file, all data stored in the HDF5 file an the corresponding database entry.
`remove_child`()	internal function to remove command that removes also child jobs.
`remove_file`()	Remove the HDF5 file with all the related content
`rename`(new_job_name)	Rename the job - by changing the job name
`reset_job_id`([job_id])	The reset_job_id function has to be implemented by the derived classes - usually the GenericJob class
`save`()	The save function has to be implemented by the derived classes - usually the GenericJob class
`self_archive`()	Compress HDF5 file of the job object to tar-archive
`self_unarchive`()	Decompress HDF5 file of the job object from tar-archive
`show_hdf`()	Iterating over the HDF5 datastructure and generating a human readable graph.
`to_hdf`([hdf, group_name])	Store object in hdf5 format - The function has to be implemented by the derived classes - usually the GenericJob class
`to_object`([object_type])	Load the full pyiron object from an HDF5 file
`values`()	List all values for all groups and nodes of the HDF5 file

Attributes

`base_name`	Name of the HDF5 file - but without the file extension .h5
`child_ids`	list of child job ids - only meta jobs have child jobs - jobs which list the meta job as their master
`content`
`database_entry`
`file_path`	Path where the HDF5 file is located - posixpath.dirname()
`files`	Allows to browse the files in a job directory.
`files_to_compress`
`files_to_remove`
`h5_path`	Get the path in the HDF5 file starting from the root group - meaning this path starts with '/'
`id`	Unique id to identify the job in the pyiron database - use self.job_id instead
`is_empty`	Check if the HDF5 file is empty
`is_root`	Check if the current h5_path is pointing to the HDF5 root group.
`job_id`	Unique id to identify the job in the pyiron database
`job_info_str`	Short string to describe the job by it is job_name and job ID - mainly used for logging
`job_name`	Get name of the job, which has to be unique within the project
`master_id`	Get job id of the master job - a meta job which groups a series of jobs, which are executed either in parallel or in serial.
`name`	Get name of the job, which has to be unique within the project
`parent_id`	Get job id of the predecessor job - the job which was executed before the current one in the current job series
`path`	Absolute path of the HDF5 group starting from the system root - combination of the absolute system path plus the absolute path inside the HDF5 file starting from the root group.
`project`	Project instance the jobs is located in
`project_hdf5`	Get the ProjectHDFio instance which points to the HDF5 file the job is stored in
`status`	Execution status of the job, can be one of the following [initialized, appended, created, submitted, running,
`working_directory`	working directory of the job is executed in - outside the HDF5 file

property base_name: str#

Name of the HDF5 file - but without the file extension .h5

Returns:: file name without the file extension
Return type:: str

check_if_job_exists(job_name: str | None = None, project: ProjectHDFio | pyiron_base.project.generic.Project | None = None)#

Check if a job already exists in an specific project.

Parameters:

job_name (str) – Job name (optional)
project (ProjectHDFio, Project) – Project path (optional)

Returns:

True / False

Return type:

(bool)

property child_ids: list#

list of child job ids - only meta jobs have child jobs - jobs which list the meta job as their master

Returns:: list of child job ids
Return type:: list

close() → None[source]#: Close the current HDF5 path and return to the path before the last open

compress(files_to_compress: List[str] | None = None, files_to_remove: List[str] | None = None) → None#

Compress the output files of a job object.

Parameters:: files_to_compress (list)

copy() → JobCore#

Copy the JobCore object which links to the HDF5 file

Returns:: New FileHDFio object pointing to the same HDF5 file
Return type:: JobCore

copy_to(project: JobCore | ProjectHDFio | pyiron_base.project.generic.Project, new_job_name: str | None = None, input_only: bool = False, new_database_entry: bool = True, copy_files: bool = True) → JobCore#

Copy the content of the job including the HDF5 file to a new location

Parameters:

project (JobCore/ProjectHDFio/Project) – project to copy the job to
new_job_name (str) – The new name to assign the duplicate job. Required if the project is None or the same project as the copied job. (Default is None, try to keep the same name.)
input_only (bool) – [True/False] Whether to copy only the input. (Default is False.)
new_database_entry (bool) – [True/False] Whether to create a new database entry. If input_only is True then new_database_entry is False. (Default is True.)
copy_files (bool) – [True/False] copy the files inside the working directory - default True

Returns:

JobCore object pointing to the new location.

Return type:

JobCore

create_group(name: str) → pyiron_base.storage.hdfio.ProjectHDFio[source]#

Create an HDF5 group - similar to a folder in the filesystem - the HDF5 groups allow the users to structure their data.

Parameters:: name (str) – name of the HDF5 group
Returns:: FileHDFio object pointing to the new group
Return type:: FileHDFio

decompress() → None#: Decompress the output files of a compressed job object.

property file_path: str#

Path where the HDF5 file is located - posixpath.dirname()

Returns:: HDF5 file location
Return type:: str

property files: FileBrowser#

Allows to browse the files in a job directory.

By default this object prints itself as a listing of the job directory and the files inside.
>>> job.files
/path/to/my/job:
    pyiron.log
    error.out
Access to the names of files is provided with list()
>>> job.files.list()
['pyiron.log', 'error.out', 'INCAR']
Access to the contents of files is provided by indexing into this object, which returns a list of lines in the file
>>> job.files['error.out']
["Oh no

“, “Something went wrong! “]

The tail() method prints the last lines of a file to stdout
>>> job.files.tail('error.out', lines=1)
Something went wrong!
For files that have valid python variable names can also be accessed by attribute notation
>>> job.files.INCAR 
File('INCAR')

classmethod from_db_entry(db_entry: dict) → JobPath[source]#

Load a job path from a database entry.

Parameters:: db_entry (dict) – database entry {“job”:, “subjob”:, “projectpath”:, “project”:, “hamilton”:, “hamversion”:, “status”:} and optional entries are {“id”:, “masterid”:, “parentid”:}

from_hdf(hdf: ProjectHDFio | None = None, group_name: str = 'group') → None#

Restore object from hdf5 format - The function has to be implemented by the derived classes - usually the GenericJob class

Parameters:

hdf (ProjectHDFio) – Optional hdf5 file, otherwise self is used.
group_name (str) – Optional hdf5 group in the hdf5 file.

classmethod from_job_id(db, job_id: int) → JobPath[source]#

Load a job path from a database connection and the job id.

Parameters:

db (DatabaseAccess) – database connection
job_id (int) – Job ID in the database

get(name: str, default: Any | None = None) → Any#

Internal wrapper function for __getitem__() - self[name]

Parameters:

key (str, slice) – path to the data or key of the data object
default (any, optional) – return this if key cannot be found

Returns:

data or data object

Return type:

dict, list, float, int

Raises:

ValueError – key cannot be found and default is not given

get_from_table(path: str, name: str) → dict | list | float | int#

Get a specific value from a pandas.Dataframe

Parameters:

path (str) – relative path to the data object
name (str) – parameter key

Returns:

the value associated to the specific parameter key

Return type:

dict, list, float, int

get_job_id(job_specifier: int | str | None = None) → int | None#

get the job_id for job named job_name in the local project path from database

Parameters:: job_specifier (str, int) – name of the job or job ID
Returns:: job ID of the job
Return type:: int

groups() → list[source]#

Filter HDF5 file by groups

Returns:: an HDF5 file which is filtered by groups
Return type:: FileHDFio

property h5_path: str#

Get the path in the HDF5 file starting from the root group - meaning this path starts with ‘/’

Returns:: HDF5 path
Return type:: str

property id: int#

Unique id to identify the job in the pyiron database - use self.job_id instead

Returns:: job id
Return type:: int

inspect(job_specifier: str | int) → JobCore#

Inspect an existing pyiron object - most commonly a job - from the database

Parameters:: job_specifier (str, int) – name of the job or job ID
Returns:: Access to the HDF5 object - not a GenericJob object - use load() instead.
Return type:: JobCore

is_compressed() → bool#

Check if the job is already compressed or not.

Returns:: [True/False]
Return type:: bool

property is_empty: bool#

Check if the HDF5 file is empty

Returns:: [True/False]
Return type:: bool

is_master_id(job_id: int) → bool#

Check if the job ID job_id is the master ID for any child job

Parameters:: job_id (int) – job ID of the master job
Returns:: [True/False]
Return type:: bool

property is_root: bool#

Check if the current h5_path is pointing to the HDF5 root group.

Returns:: [True/False]
Return type:: bool

is_self_archived() → bool#

Check if the HDF5 file of the Job is compressed as tar-archive

Returns:: [True/False]
Return type:: bool

items() → Tuple[list, list][source]#

List all keys and values as items of all groups and nodes of the HDF5 file

Returns:: list of sets (key, value)
Return type:: list

property job_id: int#

Unique id to identify the job in the pyiron database

Returns:: job id
Return type:: int

property job_info_str: str#

Short string to describe the job by it is job_name and job ID - mainly used for logging

Returns:: job info string
Return type:: str

property job_name: str#

Get name of the job, which has to be unique within the project

Returns:: job name
Return type:: str

keys() → list[source]#

List all groups and nodes of the HDF5 file - where groups are equivalent to directories and nodes to files.

Returns:: all groups and nodes
Return type:: list

list_all()#

Returns dictionary of :method:`.list_groups()` and :method:`.list_nodes()`.

Returns:

results of :method:`.list_groups() under the key "groups"; results of :method:`.list_nodes()` und the: key “nodes”

Return type:

dict

list_childs() → list#

List child jobs as JobPath objects - not loading the full GenericJob objects for each child

Returns:: list of child jobs
Return type:: list

list_dirs() → list[source]#

equivalent to os.listdirs (consider groups as equivalent to dirs)

Returns:: list of groups in pytables for the path self.h5_path
Return type:: (list)

list_files() → list#

List files inside the working directory

Parameters:: extension (str) – filter by a specific extension
Returns:: list of file names
Return type:: list

list_groups()#

Return a list of names of all nested groups.

Returns:: group names
Return type:: list of str

list_nodes()#

Return a list of names of all nested nodes.

Returns:: node names
Return type:: list of str

listdirs() → list[source]#

equivalent to os.listdirs (consider groups as equivalent to dirs)

Returns:: list of groups in pytables for the path self.h5_path
Return type:: (list)

load(job_specifier: str | int, convert_to_object: bool = True) → pyiron_base.job.generic.GenericJob | JobCore#

Load an existing pyiron object - most commonly a job - from the database

Parameters:

job_specifier (str, int) – name of the job or job ID
convert_to_object (bool) – convert the object to an pyiron object or only access the HDF5 file - default=True accessing only the HDF5 file is about an order of magnitude faster, but only provides limited functionality. Compare the GenericJob object to JobCore object.

Returns:

Either the full GenericJob object or just a reduced JobCore object

Return type:

GenericJob, JobCore

property master_id: int#

Get job id of the master job - a meta job which groups a series of jobs, which are executed either in parallel or in serial.

Returns:: master id
Return type:: int

move_to(project: ProjectHDFio) → None#

Move the content of the job including the HDF5 file to a new location

Parameters:: project (ProjectHDFio) – project to move the job to

property name: str#

Get name of the job, which has to be unique within the project

Returns:: job name
Return type:: str

nodes() → list[source]#

Filter HDF5 file by nodes

Returns:: an HDF5 file which is filtered by nodes
Return type:: FileHDFio

open(h5_rel_path: str) → pyiron_base.storage.hdfio.ProjectHDFio[source]#

Create an HDF5 group and enter this specific group. If the group exists in the HDF5 path only the h5_path is set correspondingly otherwise the group is created first.

Parameters:: h5_rel_path (str) – relative path from the current HDF5 path - h5_path - to the new group
Returns:: FileHDFio object pointing to the new group
Return type:: FileHDFio

property parent_id: int#

Get job id of the predecessor job - the job which was executed before the current one in the current job series

Returns:: parent id
Return type:: int

property path: str#

Absolute path of the HDF5 group starting from the system root - combination of the absolute system path plus the absolute path inside the HDF5 file starting from the root group.

Returns:: absolute path
Return type:: str

property project: pyiron_base.project.generic.Project#

Project instance the jobs is located in

Returns:: project the job is located in
Return type:: Project

property project_hdf5: ProjectHDFio#

Get the ProjectHDFio instance which points to the HDF5 file the job is stored in

Returns:: HDF5 project
Return type:: ProjectHDFio

put(key: str, value: Any) → None[source]#

Store data inside the HDF5 file

Parameters:

key (str) – key to store the data
value (pandas.DataFrame, pandas.Series, dict, list, float, int) – basically any kind of data is supported

relocate_hdf5(h5_path: str | None = None)#: Relocate the hdf file. This function is needed when the child job is spawned by a parent job (cf. pyiron_base.jobs.master.generic)

remove(_protect_childs: bool = True) → None#

Remove the job - this removes the HDF5 file, all data stored in the HDF5 file an the corresponding database entry.

Parameters:: _protect_childs (bool) – [True/False] by default child jobs can not be deleted, to maintain the consistency - default=True

remove_child() → None#: internal function to remove command that removes also child jobs. Do never use this command, since it will destroy the integrity of your project.

remove_file() → None[source]#: Remove the HDF5 file with all the related content

rename(new_job_name: str) → None#

Rename the job - by changing the job name

Parameters:: new_job_name (str) – new job name

reset_job_id(job_id: int | None = None) → None#

The reset_job_id function has to be implemented by the derived classes - usually the GenericJob class

Parameters:: job_id (int/ None)

save() → None#: The save function has to be implemented by the derived classes - usually the GenericJob class

self_archive() → None#: Compress HDF5 file of the job object to tar-archive

self_unarchive() → None#: Decompress HDF5 file of the job object from tar-archive

show_hdf() → None#: Iterating over the HDF5 datastructure and generating a human readable graph.

property status: str#

Execution status of the job, can be one of the following [initialized, appended, created, submitted, running,: aborted, collect, suspended, refresh, busy, finished]

Returns:: status
Return type:: (str/pyiron_base.job.jobstatus.JobStatus)

to_hdf(hdf: ProjectHDFio | None = None, group_name: str = 'group') → None#

Store object in hdf5 format - The function has to be implemented by the derived classes - usually the GenericJob class

Parameters:

hdf (ProjectHDFio) – Optional hdf5 file, otherwise self is used.
group_name (str) – Optional hdf5 group in the hdf5 file.

to_object(object_type: str | None = None, **qwargs) → pyiron_base.job.generic.GenericJob#

Load the full pyiron object from an HDF5 file

Parameters:

object_type – if the ‘TYPE’ node is not available in the HDF5 file a manual object type can be set - optional
**qwargs – optional parameters [‘job_name’, ‘project’] - to specify the location of the HDF5 path

Returns:

pyiron object

Return type:

GenericJob

values() → list[source]#

List all values for all groups and nodes of the HDF5 file

Returns:: list of all values
Return type:: list

property working_directory: str#

working directory of the job is executed in - outside the HDF5 file

Returns:: working directory
Return type:: str

pyiron_base.jobs.job.path.JobPath

Contents

pyiron_base.jobs.job.path.JobPath#