wfcommons.wfchef

wfcommons.wfchef.chef

wfcommons.wfchef.wfchef_abstract_recipe

class wfcommons.wfchef.wfchef_abstract_recipe.BaseMethod(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

BIGGEST = 2

ERROR_TABLE = 0

RANDOM = 3

SMALLEST = 1

class wfcommons.wfchef.wfchef_abstract_recipe.WfChefWorkflowRecipe(name: str, data_footprint: int | None, num_tasks: int | None, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0, logger: Logger | None = None, this_dir: str | Path = None, base_method: Enum | None = BaseMethod.ERROR_TABLE)

Bases: WorkflowRecipe

An abstract class of workflow recipes for creating synthetic workflow instances.

Parameters:

name (str) – The workflow recipe name.
data_footprint (int) – The upper bound for the workflow total data footprint (in bytes).
num_tasks (int) – The upper bound for the total number of tasks in the workflow.
runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.
logger (Logger) – The logger where to log information/warning or errors (optional).

_abc_impl = <_abc._abc_data object>

_load_base_graph() → DiGraph

_load_microstructures() → Dict

_workflow_recipe() → Dict[str, Any]

Recipe for generating synthetic instances for a workflow. Recipes can be generated by using the InstanceAnalyzer.

Returns:: A recipe in the form of a dictionary in which keys are task prefixes.
Return type:: Dict[str, Any]

build_workflow(workflow_name: str | None = None) → Workflow

Generate a synthetic workflow instance.

Parameters:: workflow_name (int) – The workflow name
Returns:: A synthetic workflow instance object.
Return type:: Workflow

classmethod from_num_tasks(num_tasks: int, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0) → WfChefWorkflowRecipe

Instantiate a workflow recipe that will generate synthetic workflows up to the total number of tasks provided.

Parameters:

num_tasks (int) – The upper bound for the total number of tasks in the workflow.
exclude_graphs (Set) –
runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.

Returns:

A workflow recipe object that will generate synthetic workflows up to the total number of tasks provided.

Return type:

WfChefWorkflowRecipe

generate_nx_graph() → DiGraph

wfcommons.wfchef.duplicate

exception wfcommons.wfchef.duplicate.NoMicrostructuresError: Bases: Exception

wfcommons.wfchef.duplicate.duplicate(path: Path, base: str | Path, num_nodes: int) → DiGraph

Attaches replicated nodes to base graph.

Parameters:

path (pathlib.Path.) – path to the summary JSON file.
base (str or pathlib.Path.) – name (for samples available in WfCommons) or path to the specific graph to be used as base (if not set WfChef chooses the best fitting one).
num_nodes (int.) – total amount of nodes desired in the synthetic instance.

Returns:

graph with the desired number of tasks.

Return type:

networkX DiGraph.

wfcommons.wfchef.duplicate.duplicate_nodes(graph: DiGraph, nodes: Set[str]) → Dict

Replicates nodes of a graph.

Parameters:

graph (networkX DiGraph) – graph used to replicate and attach new nodes.
nodes (Set[str].) – nodes to be replicated.

Returns:

the new nodes replicated.

Return type:

Dict[str].

wfcommons.wfchef.find_microstructures

wfcommons.wfchef.utils

wfcommons.wfchef.utils.annotate(g: DiGraph) → None

Annotates a networkX DiGraph with metadata such as the tasks top-down type hash, bottom-up type hash, and type-hash.

Parameters:: path (str or pathlib.Path.) – name (for samples available in WfCommons) or the path to graphs JSON.
Returns:: annotated graph.
Return type:: networkX DiGraph.

wfcommons.wfchef.utils.combine_hashes(*hashes: str) → str

wfcommons.wfchef.utils.create_graph(path: Path) → DiGraph

Creates a networkX DiGraph from a JSON file in the WfFormat.

Parameters:: path (pathlib.Path) – name (for samples available in WfCommons) or the path to graphs JSON.
Returns:: graph.
Return type:: networkX DiGraph.

wfcommons.wfchef.utils.draw(g: DiGraph, extension: str | None = 'png', with_labels: bool = False, ax: Axes | None = None, show: bool = False, save: Path | str | None = None, close: bool = False, legend: bool = False, node_size: int = 1000, linewidths: int = 5, subgraph: Set[str] = {}) → Tuple[Figure, Axes]

Plots a netwrokX DiGraph.

Parameters:

g (networkX DiGraph.) – graph to be plotted.
extension (extension of the output file.) – str.
with_labels (bool.) – if set, it prints the task types over their nodes.
ax (plt.Axes.) – plot axes.
show (bool.) – if set, displays the plot on screen.
save (pathlib.Path.) – path to directory to save the plot.
close (bool.) – if set, automatically closes window that displays plot.
legend (bool.) – if set, displays legend of the plot.
node_size (int.) – size of the nodes (circles) in the plot.
linewidths (int.) – thickness of the edges in the plot.
subgraph (Set[str].) – nodes that were added by replication and will be colored green.

Returns:

the figure and the axis used.

Return type:

Tuple[plt.Figure, plt.Axes].

wfcommons.wfchef.utils.string_hash(obj: Hashable) → str

wfcommons.wfchef.utils.type_hash(_type: str, parent_types: Iterable[str]) → str

wfcommons.wfchef.skeletons.recipe

class wfcommons.wfchef.skeletons.recipe.SkeletonRecipe(data_footprint: int | None = 0, num_tasks: int | None = 3, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0, logger: Logger | None = None, base_method: BaseMethod = BaseMethod.ERROR_TABLE, **kwargs)

Bases: WfChefWorkflowRecipe

A Skeleton workflow recipe class for creating synthetic workflow instances.

Parameters:

data_footprint (int) – The upper bound for the workflow total data footprint (in bytes).
num_tasks (int) – The upper bound for the total number of tasks in the workflow.
exclude_graphs (Set) –
runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.
logger (Logger) – The logger where to log information/warning or errors (optional).

_abc_impl = <_abc._abc_data object>