Web-tiling step fails to convert raster to Python Imaging Library image

[julietcohen]

While processing the Circum-Arctic permafrost and ground ice map, the web-tiling step failed with the error: cannot convert float NaN to integer. This occurred within viz-raster/pdgraster/WebImage.py, when we convert a raster to a Python Imaging Library image. Within to_image(), we create a no_data_mask that represents the values within the image data that have no data (0 in this case) and we replace all those values with np.nan. See the code here.

This issue is described in the layer issue in this comment.

This issue was resolved by adding a step within to_image() that converts image_data to float before we create the no_data_mask.

to_image()

    def to_image(self, image_data):
        """
            Create a PIL image from the pixel values.

            Parameters
            ----------
            pixel_values : pandas.DataFrame
                A dataframe with a pixel_row, pixel_col, and values column.

            Returns
            -------
            PIL.Image
                A PIL image.
        """

        min_val = self.min_val
        max_val = self.max_val
        nodata_val = self.nodata_val
        image_data = image_data.copy()
        rgba_list = self.rgba_list
        height = self.height
        width = self.width

        image_data = image_data.astype(float)
        no_data_mask = image_data == nodata_val

        # set the nodata value to np.nan
        if(len(no_data_mask)):
            image_data[no_data_mask] = np.nan
...

Since the solution has already been resolved, the purpose of creating this issue is to associate the bug with the viz-raster package and the branch to fix this bug. Here we can document if the added step has any unexpected consequences when processing other datasets.

[julietcohen]

The web tiling step fails at the step to_image() in WebImage.py if the nodata_val is set to None in the config. Note: for this test run, in WebImage.py, the nodata_val argument was manually set to None to match the config, because this argument's default is not set to pull from the config (yet).

in the terminal:

/home/jcohen/viz-raster/pdgraster/WebImage.py:129: RuntimeWarning: invalid value encountered in cast
  image_data_scaled = image_data_scaled.astype(int)

and in the log:

ERROR:logger:Error creating web tile for tile Tile(x=2, y=0, z=1) from 
ERROR:logger:list index out of range

A deeper look into the error reveals that it occurs at the specified line because there are still remaining np.nan values in image_data_scaled after the cell values are scaled and the nodata_mask values are converted to 256. There are no None values.

[julietcohen]

The above comment is also posted in issue#16. Both issues identify ways to improve to_image() in WebImage.py. The fact that to_image() fails based on what values is set to the nodata_val is not because of the float fix outlined in the initial comment for this issue. However, the float fix is not the only fix/additional needed to make to_image() run smoothly with different dataset inputs and config options.

[robyngit]

It looks like this was fixed and merged into both develop and main with PR #23. We should test that the issue is fixed then close this.