shok.patch package

class shok.patch.ObjectDetectionPatch(model, patch_shape: tuple[int, int, int] = (3, 2048, 2048), learning_rate: float = 20.0, targeted: bool = False, verbose: bool = True, clip_values: tuple[float, float] | None = (0, 255), use_y_hat: bool = False, gamma: float = 0.995, patch_combiner: Module | None = None, val_patch_combiner: Module | None = None, patched_image_transforms: Module | None = None, val_patched_image_transforms: Module | None = None, eot_samples: int = 1, eot_rate: int = 32)

Bases: LightningModule

ObjectDetectionPatch is a PyTorch Lightning module designed for adversarial patch training in object detection models.

A PyTorch Lightning module for adversarial patch training in object detection models. This module manages the creation, optimization, and application of an adversarial patch to input images, with the goal of affecting object detection performance. It supports expectation over transformation (EOT) sampling, custom image mutators, gradient accumulation, and evaluation using mean average precision (mAP).

Notes:

• The module is designed for adversarial patch training and evaluation in object detection tasks.

• Manual optimization is used for fine-grained control over patch updates.

• Logging is performed for patch statistics, gradients, and evaluation metrics.

• The patch is clipped to specified value ranges after each update.

configure_optimizers()

Configure the optimizer for the patch.

eot_sample_batches(batch)

Generates multiple transformed batches using EOT (Expectation Over Transformation) sampling.

Args:

batch (tuple): A tuple containing two lists:

• batch[0]: List of input images.

• batch[1]: List of corresponding targets.

Yields:

tuple: A tuple of two lists:

• The first list contains transformed images.

• The second list contains transformed targets.

Each yielded batch is created by applying self.combiner and self.other_transforms to each image-target pair in the input batch, repeated self.eot_samples times.

on_train_epoch_end()

Called at the end of each training epoch.

Increments the eot_samples counter every 16 epochs and logs its value. The logging is performed at the end of each epoch and displayed in the progress bar.

Side Effects:

• Updates self.eot_samples if the current epoch is a multiple of 16.

• Logs the value of eot_samples with the key “eot_samples”.

on_train_epoch_start()

Called at the start of each training epoch.

Ensures the module is set to training mode by calling self.train(). Note: This method currently requires manual invocation to set the mode.

on_validation_epoch_start()

Called at the start of the validation epoch.

Sets the model to evaluation mode by disabling training-specific behaviors such as dropout and batch normalization updates.

train(mode=True)

Sets the module and its underlying model to training or evaluation mode.

Args:

mode (bool, optional): If True, sets the module to training mode;

if False, sets to evaluation mode. Defaults to True.

Returns:

self: Returns the module instance.

Notes:

• Calls the superclass’s train method.

• Sets the underlying model to train or eval mode accordingly.

• For certain normalization and attention layers, forces evaluation mode and disables gradient updates for their weights and biases when

in training mode. - Contains commented-out code and TODOs for future cleanup.

training_step(batch, batch_idx)

Performs a training step over a batch of data, applying multiple EOT (Expectation Over Transformation) samples per batch.

For each EOT sample generated from the input batch, computes the model losses, logs individual loss components, aggregates the total loss, and applies manual backward propagation with loss scaling. Loss is scaled by the number of EOT samples and the total number of training batches to account for gradient accumulation. Triggers model update if the batch index meets update criteria.

Args:

batch: The input batch of data. batch_idx (int): The index of the current batch.

Returns:

None

validation_step(batch, batch_idx, dataloader_idx)

Validation step for the patch.

Submodules

shok.patch.module module

class shok.patch.module.ObjectDetectionPatch(model, patch_shape: tuple[int, int, int] = (3, 2048, 2048), learning_rate: float = 20.0, targeted: bool = False, verbose: bool = True, clip_values: tuple[float, float] | None = (0, 255), use_y_hat: bool = False, gamma: float = 0.995, patch_combiner: Module | None = None, val_patch_combiner: Module | None = None, patched_image_transforms: Module | None = None, val_patched_image_transforms: Module | None = None, eot_samples: int = 1, eot_rate: int = 32)

Bases: LightningModule

ObjectDetectionPatch is a PyTorch Lightning module designed for adversarial patch training in object detection models.

A PyTorch Lightning module for adversarial patch training in object detection models. This module manages the creation, optimization, and application of an adversarial patch to input images, with the goal of affecting object detection performance. It supports expectation over transformation (EOT) sampling, custom image mutators, gradient accumulation, and evaluation using mean average precision (mAP).

Notes:

• The module is designed for adversarial patch training and evaluation in object detection tasks.

• Manual optimization is used for fine-grained control over patch updates.

• Logging is performed for patch statistics, gradients, and evaluation metrics.

• The patch is clipped to specified value ranges after each update.

configure_optimizers()

Configure the optimizer for the patch.

eot_sample_batches(batch)

Generates multiple transformed batches using EOT (Expectation Over Transformation) sampling.

Args:

batch (tuple): A tuple containing two lists:

• batch[0]: List of input images.

• batch[1]: List of corresponding targets.

Yields:

tuple: A tuple of two lists:

• The first list contains transformed images.

• The second list contains transformed targets.

Each yielded batch is created by applying self.combiner and self.other_transforms to each image-target pair in the input batch, repeated self.eot_samples times.

on_train_epoch_end()

Called at the end of each training epoch.

Increments the eot_samples counter every 16 epochs and logs its value. The logging is performed at the end of each epoch and displayed in the progress bar.

Side Effects:

• Updates self.eot_samples if the current epoch is a multiple of 16.

• Logs the value of eot_samples with the key “eot_samples”.

on_train_epoch_start()

Called at the start of each training epoch.

Ensures the module is set to training mode by calling self.train(). Note: This method currently requires manual invocation to set the mode.

on_validation_epoch_start()

Called at the start of the validation epoch.

Sets the model to evaluation mode by disabling training-specific behaviors such as dropout and batch normalization updates.

train(mode=True)

Sets the module and its underlying model to training or evaluation mode.

Args:

mode (bool, optional): If True, sets the module to training mode;

if False, sets to evaluation mode. Defaults to True.

Returns:

self: Returns the module instance.

Notes:

• Calls the superclass’s train method.

• Sets the underlying model to train or eval mode accordingly.

• For certain normalization and attention layers, forces evaluation mode and disables gradient updates for their weights and biases when

in training mode. - Contains commented-out code and TODOs for future cleanup.

training_step(batch, batch_idx)

Performs a training step over a batch of data, applying multiple EOT (Expectation Over Transformation) samples per batch.

For each EOT sample generated from the input batch, computes the model losses, logs individual loss components, aggregates the total loss, and applies manual backward propagation with loss scaling. Loss is scaled by the number of EOT samples and the total number of training batches to account for gradient accumulation. Triggers model update if the batch index meets update criteria.

Args:

batch: The input batch of data. batch_idx (int): The index of the current batch.

Returns:

None

validation_step(batch, batch_idx, dataloader_idx)

Validation step for the patch.