wfcommons.wfgen
wfcommons.wfgen.generator
- class wfcommons.wfgen.generator.WorkflowGenerator(workflow_recipe: wfcommons.wfgen.abstract_recipe.WorkflowRecipe, logger: Optional[logging.Logger] = None)
Bases:
object
A generator of synthetic workflow instances based on workflow recipes obtained from the analysis of real workflow execution instances.
- Parameters
workflow_recipe (WorkflowRecipe) – The workflow recipe to be used for this generator.
logger (Optional[Logger]) – The logger where to log information/warning or errors (optional).
- build_workflow(workflow_name: Optional[str] = None) wfcommons.common.workflow.Workflow
Generate a synthetic workflow instance based on the workflow recipe used to instantiate the generator.
- Parameters
workflow_name (Optional[str]) – The workflow name.
- Returns
A synthetic workflow instance object.
- Return type
- build_workflows(num_workflows: int) List[wfcommons.common.workflow.Workflow]
Generate a number of synthetic workflow instances based on the workflow recipe used to instantiate the generator.
- Parameters
num_workflows (int) – The number of workflows to be generated.
- Returns
A list of synthetic workflow instance objects.
- Return type
List[Workflow]
wfcommons.wfgen.abstract_recipe
- class wfcommons.wfgen.abstract_recipe.WorkflowRecipe(name: str, data_footprint: Optional[int], num_tasks: Optional[int], runtime_factor: Optional[float] = 1.0, input_file_size_factor: Optional[float] = 1.0, output_file_size_factor: Optional[float] = 1.0, logger: Optional[logging.Logger] = None)
Bases:
abc.ABC
An abstract class of workflow recipes for creating synthetic workflow instances.
- Parameters
name (str) – The workflow recipe name.
data_footprint (Optional[int]) – The upper bound for the workflow total data footprint (in bytes).
num_tasks (Optional[int]) – The upper bound for the total number of tasks in the workflow.
runtime_factor (Optional[float]) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (Optional[float]) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (Optional[float]) – The factor of which tasks output files size will be increased/decreased.
logger (Optional[Logger]) – The logger where to log information/warning or errors (optional).
- _abc_impl = <_abc_data object>
- _generate_file(extension: str, recipe: Dict[str, Any], link: wfcommons.common.file.FileLink) wfcommons.common.file.File
Generate a file according to a file recipe.
- _generate_files(task_id: str, recipe: Dict[str, Any], link: wfcommons.common.file.FileLink) List[wfcommons.common.file.File]
Generate files for a specific task ID.
- _generate_task(task_name: str, task_id: str) wfcommons.common.task.Task
Generate a synthetic task.
- Parameters
task_name (str) – task name.
task_id (str) – task ID.
- Returns
A task object.
- Return type
task
- _generate_task_files(task: wfcommons.common.task.Task) List[wfcommons.common.file.File]
Generate input and output files for a task.
- _generate_task_name(prefix: str) str
Generate a task name from a prefix appended with an ID.
- Parameters
prefix (str) – task prefix.
- Returns
task name from prefix appended with an ID.
- Return type
str
- _get_files_by_task_and_link(task_id: str, link: wfcommons.common.file.FileLink) List[wfcommons.common.file.File]
Get the list of files for a task ID and link type.
- abstract _workflow_recipe() Dict[str, Any]
Recipe for generating synthetic instances for a workflow. Recipes can be generated by using the
InstanceAnalyzer
.- Returns
A recipe in the form of a dictionary in which keys are task prefixes.
- Return type
Dict[str, Any]
- abstract build_workflow(workflow_name: Optional[str] = None) wfcommons.common.workflow.Workflow
Generate a synthetic workflow instance.
- Parameters
workflow_name (Optional[str]) – The workflow name
- Returns
A synthetic workflow instance object.
- Return type
- abstract classmethod from_num_tasks(num_tasks: int, runtime_factor: Optional[float] = 1.0, input_file_size_factor: Optional[float] = 1.0, output_file_size_factor: Optional[float] = 1.0) wfcommons.wfgen.abstract_recipe.WorkflowRecipe
Instantiate a workflow recipe that will generate synthetic workflows up to the total number of tasks provided.
- Parameters
num_tasks (int) – The upper bound for the total number of tasks in the workflow.
runtime_factor (Optional[float]) – The factor of which tasks runtime will be increased/decreased.
input_file_size_factor (Optional[float]) – The factor of which tasks input files size will be increased/decreased.
output_file_size_factor (Optional[float]) – The factor of which tasks output files size will be increased/decreased.
- Returns
A workflow recipe object that will generate synthetic workflows up to the total number of tasks provided.
- Return type
WorkflowRecipe