wfcommons.wfchef
wfcommons.wfchef.chef
wfcommons.wfchef.wfchef_abstract_recipe
- class wfcommons.wfchef.wfchef_abstract_recipe.BaseMethod(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
Bases:
Enum
- BIGGEST = 2
- ERROR_TABLE = 0
- RANDOM = 3
- SMALLEST = 1
- class wfcommons.wfchef.wfchef_abstract_recipe.WfChefWorkflowRecipe(name: str, data_footprint: int | None, num_tasks: int | None, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0, logger: Logger | None = None, this_dir: str | Path = None, base_method: Enum | None = BaseMethod.ERROR_TABLE)
Bases:
WorkflowRecipe
An abstract class of workflow recipes for creating synthetic workflow instances.
- Parameters:
name (str) – The workflow recipe name.
data_footprint (int) – The upper bound for the workflow total data footprint (in bytes).
num_tasks (int) – The upper bound for the total number of tasks in the workflow.
runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.
logger (Logger) – The logger where to log information/warning or errors (optional).
- _abc_impl = <_abc._abc_data object>
- _load_base_graph() DiGraph
- _load_microstructures() Dict
- _workflow_recipe() Dict[str, Any]
Recipe for generating synthetic instances for a workflow. Recipes can be generated by using the
InstanceAnalyzer
.- Returns:
A recipe in the form of a dictionary in which keys are task prefixes.
- Return type:
Dict[str, Any]
- build_workflow(workflow_name: str | None = None) Workflow
Generate a synthetic workflow instance.
- Parameters:
workflow_name (int) – The workflow name
- Returns:
A synthetic workflow instance object.
- Return type:
- classmethod from_num_tasks(num_tasks: int, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0) WfChefWorkflowRecipe
Instantiate a workflow recipe that will generate synthetic workflows up to the total number of tasks provided.
- Parameters:
num_tasks (int) – The upper bound for the total number of tasks in the workflow.
exclude_graphs (Set) –
runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.
- Returns:
A workflow recipe object that will generate synthetic workflows up to the total number of tasks provided.
- Return type:
WfChefWorkflowRecipe
- generate_nx_graph() DiGraph
wfcommons.wfchef.duplicate
- exception wfcommons.wfchef.duplicate.NoMicrostructuresError
Bases:
Exception
- wfcommons.wfchef.duplicate.duplicate(path: Path, base: str | Path, num_nodes: int) DiGraph
Attaches replicated nodes to base graph.
- Parameters:
path (pathlib.Path.) – path to the summary JSON file.
base (str or pathlib.Path.) – name (for samples available in WfCommons) or path to the specific graph to be used as base (if not set WfChef chooses the best fitting one).
num_nodes (int.) – total amount of nodes desired in the synthetic instance.
- Returns:
graph with the desired number of tasks.
- Return type:
networkX DiGraph.
- wfcommons.wfchef.duplicate.duplicate_nodes(graph: DiGraph, nodes: Set[str]) Dict
Replicates nodes of a graph.
- Parameters:
graph (networkX DiGraph) – graph used to replicate and attach new nodes.
nodes (Set[str].) – nodes to be replicated.
- Returns:
the new nodes replicated.
- Return type:
Dict[str].
wfcommons.wfchef.find_microstructures
wfcommons.wfchef.utils
- wfcommons.wfchef.utils.annotate(g: DiGraph) None
Annotates a networkX DiGraph with metadata such as the tasks top-down type hash, bottom-up type hash, and type-hash.
- Parameters:
path (str or pathlib.Path.) – name (for samples available in WfCommons) or the path to graphs JSON.
- Returns:
annotated graph.
- Return type:
networkX DiGraph.
- wfcommons.wfchef.utils.combine_hashes(*hashes: str) str
- wfcommons.wfchef.utils.create_graph(path: Path) DiGraph
Creates a networkX DiGraph from a JSON file in the WfFormat.
- Parameters:
path (pathlib.Path) – name (for samples available in WfCommons) or the path to graphs JSON.
- Returns:
graph.
- Return type:
networkX DiGraph.
- wfcommons.wfchef.utils.draw(g: DiGraph, extension: str | None = 'png', with_labels: bool = False, ax: Axes | None = None, show: bool = False, save: Path | str | None = None, close: bool = False, legend: bool = False, node_size: int = 1000, linewidths: int = 5, subgraph: Set[str] = {}) Tuple[Figure, Axes]
Plots a netwrokX DiGraph.
- Parameters:
g (networkX DiGraph.) – graph to be plotted.
extension (extension of the output file.) – str.
with_labels (bool.) – if set, it prints the task types over their nodes.
ax (plt.Axes.) – plot axes.
show (bool.) – if set, displays the plot on screen.
save (pathlib.Path.) – path to directory to save the plot.
close (bool.) – if set, automatically closes window that displays plot.
legend (bool.) – if set, displays legend of the plot.
node_size (int.) – size of the nodes (circles) in the plot.
linewidths (int.) – thickness of the edges in the plot.
subgraph (Set[str].) – nodes that were added by replication and will be colored green.
- Returns:
the figure and the axis used.
- Return type:
Tuple[plt.Figure, plt.Axes].
- wfcommons.wfchef.utils.string_hash(obj: Hashable) str
- wfcommons.wfchef.utils.type_hash(_type: str, parent_types: Iterable[str]) str
wfcommons.wfchef.skeletons.recipe
- class wfcommons.wfchef.skeletons.recipe.SkeletonRecipe(data_footprint: int | None = 0, num_tasks: int | None = 3, exclude_graphs: Set[str] = {}, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0, logger: Logger | None = None, base_method: BaseMethod = BaseMethod.ERROR_TABLE, **kwargs)
Bases:
WfChefWorkflowRecipe
A Skeleton workflow recipe class for creating synthetic workflow instances.
- Parameters:
data_footprint (int) – The upper bound for the workflow total data footprint (in bytes).
num_tasks (int) – The upper bound for the total number of tasks in the workflow.
exclude_graphs (Set) –
runtime_factor (float) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (float) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (float) – The factor of which tasks output files size will be increased/decreased.
logger (Logger) – The logger where to log information/warning or errors (optional).
- _abc_impl = <_abc._abc_data object>