Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Airbyte local preq documentation. Remove outdated Gitbook section. #2274

Merged
merged 4 commits into from
Mar 3, 2021

Conversation

davinchia
Copy link
Contributor

@davinchia davinchia commented Mar 3, 2021

What

  • Add some missing prerequisites for Airbyte local development.
  • Remove the Gitbook section on updating documentation. Gitbook's editor does funky autoformatting.

Pre-merge Checklist

  • Run integration tests
  • Publish Docker images

4. `Docker`
5. `Postgresql`
6. `Jq`
7. `CMake`i
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a typo with extra "i"?

I'm not sure I have "cmake" installed (but maybe "make" through XCode installation I guess...)

Copy link
Contributor Author

@davinchia davinchia Mar 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops. this often happens to me when I'm on the butterfly keyboard. good catch.

interestingly, I explicitly had to install cmake for this to work. let me see if I can reproduce my error.

Copy link
Contributor Author

@davinchia davinchia Mar 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

brew uninstall cmake results in this error for me:

  copying pyarrow/tests/data/parquet/v0.7.1.some-named-index.parquet -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/data/parquet
  creating build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/common.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/conftest.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_basic.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_data_types.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_dataset.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_datetime.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_metadata.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_pandas.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_parquet_file.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  copying pyarrow/tests/parquet/test_parquet_writer.py -> build/lib.macosx-11.2-x86_64-3.7/pyarrow/tests/parquet
  running build_ext
  creating /private/var/folders/4h/k452x8ns06q7w_w7wkb3j16h0000gn/T/pip-install-cijvebo1/pyarrow/build/temp.macosx-11.2-x86_64-3.7
  -- Running cmake for pyarrow
  cmake -DPYTHON_EXECUTABLE=/Users/davinchia/.pyenv/versions/3.7.9/bin/python3.7 -DPython3_EXECUTABLE=/Users/davinchia/.pyenv/versions/3.7.9/bin/python3.7  -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /private/var/folders/4h/k452x8ns06q7w_w7wkb3j16h0000gn/T/pip-install-cijvebo1/pyarrow
  error: command 'cmake' failed with exit status 1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, then it's related to pyarrow, since on my machine, I don't rebuild it completely but managed to pip install pyarrow binaries directly... but it's true I haven't tried using parquet yet so it might not work... I don't know 😆

@davinchia davinchia marked this pull request as ready for review March 3, 2021 10:07
@davinchia davinchia merged commit 2ac9d36 into master Mar 3, 2021
@davinchia davinchia deleted the update-local-dev-prereq branch March 3, 2021 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants