Registration fails for simple example (rotating the same image)

[idc9]

I'm having difficulty getting pystackreg to work for a simple example where I rotate the same square image by 90 degrees and try to register the rotation back to the original.

I'm using pystackreg version 0.2.7, python 3.8.12, and a macOS-10.16-x86_64-i386-64bit

I'm having similar issues on other images (all from histopathology). I've tried a few simple things like binarizing the images (i.e. tissue vs. background) or changing the datatypes (np.uint8, float), but nothing helps. My issue is perhaps similar to this issue.

Reproducible example

Here is the original RGB image so you can run the below code yourself

Some setup

from skimage.transform import rotate
from skimage.color import rgb2gray
from skimage.io import imread
from skimage.exposure import equalize_hist

import matplotlib.pyplot as plt
import numpy as np

from pystackreg import StackReg
import pystackreg

save_dir = './'

print("pystackreg version", pystackreg.__version__)

def composite_images(imgs, equalize=False):
    if equalize:
        imgs = [exposure.equalize_hist(img) for img in imgs]
    
    imgs = [img / img.max() for img in imgs]
    
    if len(imgs) < 3:
        imgs += [np.zeros(shape=imgs[0].shape)] * (3-len(imgs))
  
    imgs = np.dstack(imgs)
    
    return imgs

def plot_ctsas(target, source_aligned, source, equalize=False):
    """
    Plots the composite, target, source aligned, and source image.
    """
    composite = composite_images([target, source_aligned], equalize=equalize)
    
    
    plt.subplot(1, 4, 1)
    plt.imshow(composite)
    plt.axis('off')
    plt.title("composite")
    
    plt.subplot(1, 4, 2)
    plt.imshow(target, cmap='gray')
    plt.axis('off')
    plt.title("target image")
    
    
    plt.subplot(1, 4, 3)
    plt.imshow(source_aligned, cmap='gray')
    plt.axis('off')
    plt.title("source aligned")
    
    
    plt.subplot(1, 4, 4)
    plt.imshow(source, cmap='gray')
    plt.axis('off')
    plt.title("source image")
    
   image_rgb = imread(os.path.join(save_dir, 'image.png'))

image_rotated_rgb = rotate(image=image_rgb, angle=90, preserve_range=True).astype(np.uint8)
source = rgb2gray(image_rgb)
target = rgb2gray(image_rotated_rgb)

print("original image rgb shape {}, dtype {}".format(image_rgb.shape, image_rgb.dtype))
print('source shape {}, dtype {}'.format(source.shape, source.dtype))

plt.figure(figsize=(8, 4))
plt.subplot(1, 2, 1)
plt.imshow(source, cmap='gray')
plt.title('source')
plt.axis('off')

plt.subplot(1, 2, 2)
plt.imshow(target, cmap='gray')
plt.title('target')
plt.axis('off')
plt.savefig(os.path.join(save_dir, 'source_and_target.png'), bbox_inches='tight')

pystackreg version 0.2.7
original image rgb shape (256, 256, 3), dtype uint8
source shape (256, 256), dtype float64

Try registering

Now lets try registering the rotated image

transformations = {
    'TRANSLATION': StackReg.TRANSLATION,
    'RIGID_BODY': StackReg.RIGID_BODY,
    'SCALED_ROTATION': StackReg.SCALED_ROTATION,
    'AFFINE': StackReg.AFFINE,
    'BILINEAR': StackReg.BILINEAR
}


for (name, transf) in transformations.items():
    registerer = StackReg(transf)
    source_aligned = registerer.register_transform(ref=target, mov=source)

    plt.figure(figsize=(16, 4))
    plot_ctsas(target=target, source_aligned=source_aligned, source=source)
    
    plt.savefig(os.path.join(save_dir, '{}.png'.format(name)), bbox_inches='tight')

[glichtner]

HI @idc9,

Thanks for reporting and thanks for providing the detailed example!

pystackreg (and StackReg/TurboReg, for that matter) do indeed not correctly process such strongly rotated images and I'm afraid I don't think I'm able to provide a quick methodologic fix.

I have encountered a similiar situation in a project with images that were sometimes rotated by 180° and my approach was to first determine the correlation between source and target and then between source and target-rotated-by-180° and based on these decide (automatically) whether to rotate the target image before registration.

I hope this helps. Let me know if you have a better solution.

Meanwhile, I have mentioned this issue in the README: https://github.com/glichtner/pystackreg/blob/master/README.rst#known-issues