wfcommons.wfchef

wfcommons.wfchef.chef

wfcommons.wfchef.wfchef_abstract_recipe

class wfcommons.wfchef.wfchef_abstract_recipe.BaseMethod(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

BIGGEST = 2
ERROR_TABLE = 0
RANDOM = 3
SMALLEST = 1
class wfcommons.wfchef.wfchef_abstract_recipe.WfChefWorkflowRecipe(name: str, data_footprint: int | None, num_tasks: int | None, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0, logger: Logger | None = None, this_dir: str | Path = None, base_method: Enum | None = BaseMethod.ERROR_TABLE)

Bases: WorkflowRecipe

An abstract class of workflow recipes for creating synthetic workflow instances.

Parameters:
  • name (str) – The workflow recipe name.

  • data_footprint (int) – The upper bound for the workflow total data footprint (in bytes).

  • num_tasks (int) – The upper bound for the total number of tasks in the workflow.

  • runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.

  • input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.

  • output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.

  • logger (Logger) – The logger where to log information/warning or errors (optional).

_abc_impl = <_abc._abc_data object>
_load_base_graph() DiGraph
_load_microstructures() Dict
_workflow_recipe() Dict[str, Any]

Recipe for generating synthetic instances for a workflow. Recipes can be generated by using the InstanceAnalyzer.

Returns:

A recipe in the form of a dictionary in which keys are task prefixes.

Return type:

Dict[str, Any]

build_workflow(workflow_name: str | None = None) Workflow

Generate a synthetic workflow instance.

Parameters:

workflow_name (int) – The workflow name

Returns:

A synthetic workflow instance object.

Return type:

Workflow

classmethod from_num_tasks(num_tasks: int, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0) WfChefWorkflowRecipe

Instantiate a workflow recipe that will generate synthetic workflows up to the total number of tasks provided.

Parameters:
  • num_tasks (int) – The upper bound for the total number of tasks in the workflow.

  • exclude_graphs (Set) –

  • runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.

  • input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.

  • output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.

Returns:

A workflow recipe object that will generate synthetic workflows up to the total number of tasks provided.

Return type:

WfChefWorkflowRecipe

generate_nx_graph() DiGraph

wfcommons.wfchef.duplicate

exception wfcommons.wfchef.duplicate.NoMicrostructuresError

Bases: Exception

wfcommons.wfchef.duplicate.duplicate(path: Path, base: str | Path, num_nodes: int) DiGraph

Attaches replicated nodes to base graph.

Parameters:
  • path (pathlib.Path.) – path to the summary JSON file.

  • base (str or pathlib.Path.) – name (for samples available in WfCommons) or path to the specific graph to be used as base (if not set WfChef chooses the best fitting one).

  • num_nodes (int.) – total amount of nodes desired in the synthetic instance.

Returns:

graph with the desired number of tasks.

Return type:

networkX DiGraph.

wfcommons.wfchef.duplicate.duplicate_nodes(graph: DiGraph, nodes: Set[str]) Dict

Replicates nodes of a graph.

Parameters:
  • graph (networkX DiGraph) – graph used to replicate and attach new nodes.

  • nodes (Set[str].) – nodes to be replicated.

Returns:

the new nodes replicated.

Return type:

Dict[str].

wfcommons.wfchef.find_microstructures

wfcommons.wfchef.utils

wfcommons.wfchef.utils.annotate(g: DiGraph) None

Annotates a networkX DiGraph with metadata such as the tasks top-down type hash, bottom-up type hash, and type-hash.

Parameters:

path (str or pathlib.Path.) – name (for samples available in WfCommons) or the path to graphs JSON.

Returns:

annotated graph.

Return type:

networkX DiGraph.

wfcommons.wfchef.utils.combine_hashes(*hashes: str) str
wfcommons.wfchef.utils.create_graph(path: Path) DiGraph

Creates a networkX DiGraph from a JSON file in the WfFormat.

Parameters:

path (pathlib.Path) – name (for samples available in WfCommons) or the path to graphs JSON.

Returns:

graph.

Return type:

networkX DiGraph.

wfcommons.wfchef.utils.draw(g: DiGraph, extension: str | None = 'png', with_labels: bool = False, ax: Axes | None = None, show: bool = False, save: Path | str | None = None, close: bool = False, legend: bool = False, node_size: int = 1000, linewidths: int = 5, subgraph: Set[str] = {}) Tuple[Figure, Axes]

Plots a netwrokX DiGraph.

Parameters:
  • g (networkX DiGraph.) – graph to be plotted.

  • extension (extension of the output file.) – str.

  • with_labels (bool.) – if set, it prints the task types over their nodes.

  • ax (plt.Axes.) – plot axes.

  • show (bool.) – if set, displays the plot on screen.

  • save (pathlib.Path.) – path to directory to save the plot.

  • close (bool.) – if set, automatically closes window that displays plot.

  • legend (bool.) – if set, displays legend of the plot.

  • node_size (int.) – size of the nodes (circles) in the plot.

  • linewidths (int.) – thickness of the edges in the plot.

  • subgraph (Set[str].) – nodes that were added by replication and will be colored green.

Returns:

the figure and the axis used.

Return type:

Tuple[plt.Figure, plt.Axes].

wfcommons.wfchef.utils.string_hash(obj: Hashable) str
wfcommons.wfchef.utils.type_hash(_type: str, parent_types: Iterable[str]) str

wfcommons.wfchef.skeletons.recipe

class wfcommons.wfchef.skeletons.recipe.SkeletonRecipe(data_footprint: int | None = 0, num_tasks: int | None = 3, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0, logger: Logger | None = None, base_method: BaseMethod = BaseMethod.ERROR_TABLE, **kwargs)

Bases: WfChefWorkflowRecipe

A Skeleton workflow recipe class for creating synthetic workflow instances.

Parameters:
  • data_footprint (int) – The upper bound for the workflow total data footprint (in bytes).

  • num_tasks (int) – The upper bound for the total number of tasks in the workflow.

  • exclude_graphs (Set) –

  • runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.

  • input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.

  • output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.

  • logger (Logger) – The logger where to log information/warning or errors (optional).

_abc_impl = <_abc._abc_data object>