Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove BOM in CsvFileIterator #8

Open
BenMorel opened this issue Oct 23, 2019 · 1 comment
Open

Remove BOM in CsvFileIterator #8

BenMorel opened this issue Oct 23, 2019 · 1 comment

Comments

@BenMorel
Copy link
Member

When reading a file with an UTF-8 BOM (or any other BOM, for that matter) , the BOM is part of the fgetcsv() read, which corrupts the header row:

  • the BOM is part of the first column's name
  • if an enclosure character (e.g. ") is used, it's not detected for the first column name

Note sure whether removing the BOM should be part of this library, or whether we should rely on a filter.

@BenMorel
Copy link
Member Author

Sample UTF-8 BOM filter:

class BOMFilter extends \php_user_filter
{
    private $checked = false;

    public function onCreate(): bool
    {
        return true;
    }

    public function onClose(): void
    {
    }

    public function filter($in, $out, &$consumed, $closing): int
    {
        while ($bucket = stream_bucket_make_writeable($in)) {
            if (! $this->checked) {
                if (substr($bucket->data, 0, 3) === "\xEF\xBB\xBF") {
                    $bucket->data = substr($bucket->data, 3);
                }

                $this->checked = true;
            }

            $consumed += $bucket->datalen;
            stream_bucket_append($out, $bucket);
        }

        return \PSFS_PASS_ON;
    }
}

Usage:

stream_filter_register('bom-filter', BOMFilter::class);

$fp = fopen($file, 'rb');
stream_filter_append($fp, 'bom-filter');
$rows = new CsvFileIterator($fp);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant