Blocks#

Blocks are the most important part of the HorusAPI. Blocks execute python code based on the provided PluginVariable. In order to build a Block, you first need to instantiate some PluginVariable and define the Action that the block will perform.

Warning

The PluginVariable instances are only used to model the variables of the block. In order to access the updated values, do so using the block.variables or block.variables properties.

Defining the Block#

The first step is to instantiate the Blocks class. There are different kind of blocks depending on the Action that they perform: Regular blocks (PluginBlock), Input blocks (InputBlock) and Slurm blocks (SlurmBlock).

class PluginBlock(name: str, description: str, action: Callable | None = None, variables: List[PluginVariable] = [], inputs: List[PluginVariable] = [], inputGroups: List[VariableGroup] = [], outputs: List[PluginVariable] = [], blockType: PluginBlockTypes = PluginBlockTypes.BASE, id: str | None = None, externalURL: str | None = None, category: str | None = None)#

The base block class for Horus blocks. Not to be used directly.

Initialize a PluginBlock.

class InputBlock(name, description, variable: PluginVariable, output: PluginVariable | None = None, action: Callable | None = None, id: str | None = None, externalURL: str | None = None, category: str | None = None)#

The InputBlock class is a special type of block that is used to get input from the user. It works as a regular PluginBlock but only shows its PluginVariable. Its output will be automatically set to the value the variable has if it does not have a defined action.

When only the variable parameter is defined, the block will output directly the value of the variable.

If parsing of the variable is needed, the action parameter can be used to define a function that will parse the value of the variable and return the parsed value. In that case, use the output parameter to define the output variable of the block.

Parameters:

  • name – The name of the block.

  • description – The description of the block.

  • variable – The variable of the block.

  • output – The output of the block.

  • action – The action of the block. Will be run when storing the config.

  • id – The id of the block.

  • externalURL – The external URL of the block for documentation purposes.

  • category – The category of the block inside the plugin.

class SlurmBlock(name: str, description: str, initialAction: Callable, finalAction: Callable, variables: List[PluginVariable] = [], inputs: List[PluginVariable] = [], inputGroups: List[VariableGroup] = [], outputs: List[PluginVariable] = [], id: str | None = None, failOnSlurmError: bool = True, externalURL: str | None = None, category: str | None = None)#

The SlurmBlock class is a special type of block that is used to run an action in a remote server. It works as a regular PluginBlock but it has two actions, one before the job is submitted and one after the job is completed.

Parameters:

  • name – The name of the block.

  • description – The description of the block.

  • initialAction – The action of the block before the job is submitted.

  • finalAction – The action of the block after the job is completed.

  • variables – The variables of the block.

  • inputs – The inputs of the block.

  • inputGroups – The input groups of the block.

  • outputs – The outputs of the block.

  • id – The id of the block.

  • failOnSlurmError – Whether to fail the block if the slurm job fails.

  • externalURL – The external URL of the block for documentation purposes.

PluginBlock is the most common block. It is used to execute a single python function that will be run locally on the machine. InputBlock only accepts a single PluginVariable as input and are used to pass data to the inputs of another PluginBlock or SlurmBlock. Finally, SlurmBlock executes two python functions, one before the job is sent to the a Slurm queue, and one after the job is finished.

Setting the action of the block#

The Action of the block is defined as a python function. The function must always take the block parameter with the respective PluginBlock type. If any error is raised past the Action function scope, the block will be marked as failed and the error message will be displayed in the GUI as a pop-up alert.

Accessing variables#

In order to acces the updated variables coming from the flow within the Block Action, do not use the PluginVariable object, but rather the block.variables dictionary. This dictionary contains the updated values of the variables, and can be accessed using the PluginVariable ID as the key. You can access three different kind of variables: Variables, Inputs and Configs.

def myCustomAction(block: PluginBlock):

    # The block passed to the function contains an updated dictionary of the variables
    # acces it using the variable id

    # Accessing variable value
    inputStringValue = block.variables["stringVariableID"]

    # Accessing input value
    inputFilePath = block.inputs["fileID"]

    # Accessing config value
    configValue = block.config["configID"]

    print(inputStringValue)

Setting outputs#

If your Block produces any output, you can set the value of the variable using the block.setOutput() method. As with the other variables, use the output variable ID.

def myCustomAction(block: PluginBlock):

    # Setting output value
    block.setOutput("outputVariableID", variableValue)

Examples#

Input blocks#

As an example, here is the definition of an InputBlock that simply passes a string to the next block:

String input block provided by the default Horus plugin 
from HorusAPI import PluginVariable, InputBlock, VariableTypes

# First define the variable to be used in the block
inputString = PluginVariable(
    name="String",
    id="string",
    description="A string to be used as an input.",
    type=VariableTypes.STRING,
)

# Then instantiate the block giving it a name, a description, the action to be performed and the variable.
stringBlock = InputBlock(
    name="String",
    description="A string to be used as an input.",
    action=None,
    variable=inputString,
)

As you see, if the input being passed does not required any preparation, the action can be defined as None.

Regular blocks#

Here is an example of a PluginBlock that simply prints the given input and sets it as its output:

Block that prints the input variable 
from HorusAPI import PluginVariable, PluginBlock, VariableTypes

# First define the variable to be used as input in the block
inputVariable = PluginVariable(
    name="Input",
    id="inputID",
    description="A variable to be used as an input.",
    type=VariableTypes.ANY,
)

# Define also the output variable
outputVariable = PluginVariable(
    name="Output variable",
    id="outputID",
    description="The same variable as the input.",
    type=VariableTypes.ANY,
)

# Then define the action that the block will perform
def myCustomAction(block: PluginBlock):

    # The block passed to the function contains an updated dictionary of the variables
    # acces it using the variable id
    inputValue = block.inputs["inputID"]

    print(inputValue)

    block.setOutput("outputID", inputValue)

# Finally, instantiate the block giving it a name, a description, the action to be performed and the variable.
printVariableBlock = PluginBlock(
    name="Print variable",
    description="Prints a given variable.",
    action=myCustomAction,
    inputs=[inputVariable],
    outputs=[outputVariable],
)

Slurm blocks#

Here is an example of a SlurmBlock that uploads a file to a remote server before the job is sent to the Slurm queue and downloads the result after the job is finished:

Block that sends a job to a cluster and downloads the results 
from HorusAPI import PluginVariable, SlurmBlock, VariableTypes

# First define the input to be used in the block
inputFile = PluginVariable(
    name="File",
    id="fileID",
    description="A file to upload to a remote server.",
    type=VariableTypes.FILE,
)

# Then define the output to be used in the block
outputFile = PluginVariable(
    name="Output file",
    id="outputFile",
    description="The file downloaded from the remote server after the job is finished.",
    type=VariableTypes.FILE,
)

# You can also define a regular variable, which is independent of the inputs.
# This variable will be available in the block configuration button.
regularVariable = PluginVariable(
    name="Regular variable",
    id="regularVariable",
    description="A regular variable.",
    type=VariableTypes.STRING,
)


# Then define the action that the block will perform before the job is sent to the Slurm queue
def myCustomActionBefore(block: SlurmBlock):
    # The block passed to the function contains an updated dictionary of the variables
    # acces it using the variable id either for the regular variables or the inputs.

    regularVariableValue = block.variables["regularVariable"]
    inputFilePath = block.inputs["fileID"]

    print("Regular variable value: ", regularVariableValue)

    # Upload the file to the remote server using the RemoteAPI
    block.remote.sendData(inputFilePath, "/path/in/remote/server")

    jobID = block.remote.submitJob("/path/in/remote/server")

    print("Job submitted with ID: ", jobID)


# Then define the action that the block will perform after the job is finished
def myCustomActionAfter(block: SlurmBlock):
    print("Job finished, downloading results...")

    # Download the results from the remote server using the RemoteAPI
    block.remote.getData("/path/in/remote/server", "/path/in/local/machine")

    # Set the output variable to the path of the downloaded file
    block.setOutput("outputFile", "/path/in/local/machine")


# Finally, instantiate the block giving it a name, a description, the action to be performed and the variable.
sendJobBlock = SlurmBlock(
    name="Send a job",
    description="Submits a job to the remote and downloads the results",
    initialAction=myCustomActionBefore,
    finalAction=myCustomActionAfter,
    variables=[regularVariable],
    inputs=[inputFile],
    outputs=[outputFile],
)

Adding Blocks to a Plugin#

Once you have defined several Block, you can add them to your Plugin using the .addBlock() method:

plugin.addBlock(myBlock)

Configurations#

You can add permanent variables to Block. These variables are available for modification under the Plugin configuration button and once defined, will have the same value on every run.

The PluginConfig class is just a subclass of PluginBlock, so you can define also an action which will be executed every time the configuration is modified. This can be used for configuration validation.

Instantiate the PluginConfig and then add it to the Plugin instance using the .addConfig() method.

plugin.addConfig(myConfig)

Then, the config can be accessed like the variables in the Block’s action:

myConfigValue = block.config["myConfigID"]

Storing data on blocks or flow#

Sometimes it is useful to store persistent data on a block or on the flow itself. For example, one could store the times the block has run, or a variable that is needed in the finalAction of a SlurmBlock, which is defined in the initialAction. extraData is a property of Blocks (a dictionary where to store key:value pairs) that allows developers to store variables across runs of the same block. The same applies for the extraData of a given flow.

Warning

The extraData property of a given block never gets automatically resetted, it is up to the developer of the block when to remove/overwrite the stored variables.

# Extra data of the given block
block.extraData["value_to_store"] = "myValue"

# Extra data of the flow. Not to be confused with the block.extraData property!
block.flow.extraData["my_key"] = "my_value"

Verifying if the flow was reset#

The block.dirty property is a boolean flag that indicates whether a block is being re-executed within a flow. It will be False when the block runs for the first time or after the flow has been manually reset by the user. In all subsequent executions without a reset, block.dirty will be True.

This property can be particularly useful when managing persistent data stored in extraData, as it allows the developer to distinguish between fresh executions and re-runs. For instance, if certain variables in extraData should be cleared or reset upon a flow restart, you can use block.dirty to trigger this reset manually.

if block.dirty:
    print("This block has been run multiple times without a reset...")
else:
    # Reset persistent data in extraData if the flow was reset
    block.extraData.clear()
    print("Running the block for the first time or after reset!")

This way, the developer can manage the lifecycle of extraData values, ensuring they are cleared or modified only when necessary, while maintaining persistence across non-reset runs.

Accessing the flow inside the block#

The Block has access to the instance of the Flow it is being executed in. This can be useful to access the list of blocks in the flow, the flow name, the flow ID or other properties. The flow instance can be obtained using the block.flow property.