wfcommons.wfgen
wfcommons.wfgen.generator
- class wfcommons.wfgen.generator.WorkflowGenerator(workflow_recipe: WorkflowRecipe, logger: Logger | None = None)
Bases:
object
A generator of synthetic workflow instances based on workflow recipes obtained from the analysis of real workflow execution instances.
- Parameters:
workflow_recipe (WorkflowRecipe) – The workflow recipe to be used for this generator.
logger (Optional[Logger]) – The logger where to log information/warning or errors (optional).
- build_workflow(workflow_name: str | None = None) Workflow
Generate a synthetic workflow instance based on the workflow recipe used to instantiate the generator.
- Parameters:
workflow_name (Optional[str]) – The workflow name.
- Returns:
A synthetic workflow instance object.
- Return type:
- build_workflows(num_workflows: int) List[Workflow]
Generate a number of synthetic workflow instances based on the workflow recipe used to instantiate the generator.
- Parameters:
num_workflows (int) – The number of workflows to be generated.
- Returns:
A list of synthetic workflow instance objects.
- Return type:
List[Workflow]
wfcommons.wfgen.abstract_recipe
- class wfcommons.wfgen.abstract_recipe.WorkflowRecipe(name: str, data_footprint: int | None, num_tasks: int | None, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0, logger: Logger | None = None)
Bases:
ABC
An abstract class of workflow recipes for creating synthetic workflow instances.
- Parameters:
name (str) – The workflow recipe name.
data_footprint (Optional[int]) – The upper bound for the workflow total data footprint (in bytes).
num_tasks (Optional[int]) – The upper bound for the total number of tasks in the workflow.
runtime_factor (Optional[float]) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (Optional[float]) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (Optional[float]) – The factor of which tasks output files size will be increased/decreased.
logger (Optional[Logger]) – The logger where to log information/warning or errors (optional).
- _abc_impl = <_abc._abc_data object>
- _generate_file(extension: str, recipe: Dict[str, Any], link: FileLink) File
Generate a file according to a file recipe.
- _generate_files(task_id: str, recipe: Dict[str, Any], link: FileLink) List[File]
Generate files for a specific task ID.
- _generate_task(task_name: str, task_id: str) Task
Generate a synthetic task.
- Parameters:
task_name (str) – task name.
task_id (str) – task ID.
- Returns:
A task object.
- Return type:
task
- _generate_task_name(prefix: str) str
Generate a task name from a prefix appended with an ID.
- Parameters:
prefix (str) – task prefix.
- Returns:
task name from prefix appended with an ID.
- Return type:
str
- _get_files_by_task_and_link(task_id: str, link: FileLink) List[File]
Get the list of files for a task ID and link type.
- abstract _workflow_recipe() Dict[str, Any]
Recipe for generating synthetic instances for a workflow. Recipes can be generated by using the
InstanceAnalyzer
.- Returns:
A recipe in the form of a dictionary in which keys are task prefixes.
- Return type:
Dict[str, Any]
- abstract build_workflow(workflow_name: str | None = None) Workflow
Generate a synthetic workflow instance.
- Parameters:
workflow_name (Optional[str]) – The workflow name
- Returns:
A synthetic workflow instance object.
- Return type:
- abstract classmethod from_num_tasks(num_tasks: int, runtime_factor: float | None = 1.0, input_file_size_factor: float | None = 1.0, output_file_size_factor: float | None = 1.0) WorkflowRecipe
Instantiate a workflow recipe that will generate synthetic workflows up to the total number of tasks provided.
- Parameters:
num_tasks (int) – The upper bound for the total number of tasks in the workflow.
runtime_factor (Optional[float]) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (Optional[float]) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (Optional[float]) – The factor of which tasks output files size will be increased/decreased.
- Returns:
A workflow recipe object that will generate synthetic workflows up to the total number of tasks provided.
- Return type:
WorkflowRecipe