Skip to content

Commit

Permalink
source-google-tasks contribution from btkcodedev (#45427)
Browse files Browse the repository at this point in the history
Co-authored-by: Octavia Squidington III <[email protected]>
  • Loading branch information
btkcodedev and octavia-squidington-iii authored Sep 16, 2024
1 parent a85de6f commit 37af2f0
Show file tree
Hide file tree
Showing 6 changed files with 449 additions and 0 deletions.
52 changes: 52 additions & 0 deletions airbyte-integrations/connectors/source-google-tasks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Google Tasks
This directory contains the manifest-only connector for `source-google-tasks`.

## Documentation reference:
Visit `https://developers.google.com/tasks/reference/rest` for API documentation

## Authentication setup
`Source-productive` uses bearer token authentication,
Visit `https://support.google.com/googleapi/answer/6158849?hl=en&amp;ref_topic=7013279` for getting bearer token via OAuth2.0

## Setting postman for getting bearer token
Currently Code granted OAuth 2.0 is not directly supported by airbyte, thus you could setup postman for getting the bearer token which could be used as `api_key`,
Steps:
- Visit google cloud `https://console.cloud.google.com/apis/api/tasks.googleapis.com/metrics` and enable the tasks api service
- Go to the consent screen `https://console.cloud.google.com/apis/credentials/consent` and add your email for enabling postman testing access
- Visit `https://console.cloud.google.com/apis/credentials` and create new credentails for OAuth 2.0 and copy client id and client secret
- Add callback url `https://oauth.pstmn.io/v1/callback` while credential creation
- Goto postman client and select new tab for setting authorization to OAuth 2.0
- Set scope as `https://www.googleapis.com/auth/tasks https://www.googleapis.com/auth/tasks.readonly`
- Set access token URL as `https://accounts.google.com/o/oauth2/token`
- Set auth URL as `https://accounts.google.com/o/oauth2/v2/auth`
- Click `Get New Access Token` and authorize via your google account
- Copy the resulted bearer token and use it as credential for the connector

## Usage
There are multiple ways to use this connector:
- You can use this connector as any other connector in Airbyte Marketplace.
- You can load this connector in `pyairbyte` using `get_source`!
- You can open this connector in Connector Builder, edit it, and publish to your workspaces.

Please refer to the manifest-only connector documentation for more details.

## Local Development
We recommend you use the Connector Builder to edit this connector.

But, if you want to develop this connector locally, you can use the following steps.

### Environment Setup
You will need `airbyte-ci` installed. You can find the documentation [here](airbyte-ci).

### Build
This will create a dev image (`source-google-tasks:dev`) that you can use to test the connector locally.
```bash
airbyte-ci connectors --name=source-google-tasks build
```

### Test
This will run the acceptance tests for the connector.
```bash
airbyte-ci connectors --name=source-google-tasks test
```

Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-google-tasks:dev
acceptance_tests:
spec:
tests:
- spec_path: "manifest.yaml"
connection:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
discovery:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
basic_read:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
incremental:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
full_refresh:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
6 changes: 6 additions & 0 deletions airbyte-integrations/connectors/source-google-tasks/icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
291 changes: 291 additions & 0 deletions airbyte-integrations/connectors/source-google-tasks/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,291 @@
version: 4.6.2

type: DeclarativeSource

description: >-
Website: https://tasksboard.com/app
API Docs: https://developers.google.com/tasks/reference/rest
Auth Docs:
https://support.google.com/googleapi/answer/6158849?hl=en&ref_topic=7013279
check:
type: CheckStream
stream_names:
- tasks

definitions:
streams:
tasks:
type: DeclarativeStream
name: tasks
retriever:
type: SimpleRetriever
paginator:
type: DefaultPaginator
page_token_option:
type: RequestOption
field_name: pageToken
inject_into: request_parameter
pagination_strategy:
type: CursorPagination
cursor_value: "{{ response.get('nextPageToken') }}"
stop_condition: "{{ not response.get('nextPageToken') }}"
requester:
$ref: "#/definitions/base_requester"
path: users/@me/lists
http_method: GET
error_handler:
type: CompositeErrorHandler
error_handlers:
- type: DefaultErrorHandler
max_retries: 5
response_filters:
- type: HttpResponseFilter
action: RATE_LIMITED
http_codes:
- 429
error_message: Rate limit hit
backoff_strategies:
- type: ExponentialBackoffStrategy
factor: 2
request_parameters:
maxResults: "{{ config['records_limit'] }}"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
$ref: "#/schemas/tasks"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: updated
end_datetime:
type: MinMaxDatetime
datetime: "{{ now_utc().strftime('%Y-%m-%dT%H:%M:%SZ') }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
start_datetime:
type: MinMaxDatetime
datetime: '{{ config["start_date"] }}'
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%Y-%m-%dT%H:%M:%S.%fZ"
cursor_datetime_formats:
- "%Y-%m-%dT%H:%M:%S.%fZ"
lists_tasks:
type: DeclarativeStream
name: lists_tasks
retriever:
type: SimpleRetriever
paginator:
type: DefaultPaginator
page_token_option:
type: RequestOption
field_name: pageToken
inject_into: request_parameter
pagination_strategy:
type: CursorPagination
cursor_value: "{{ response.get('nextPageToken') }}"
stop_condition: "{{ not response.get('nextPageToken') }}"
requester:
$ref: "#/definitions/base_requester"
path: lists/{{ stream_partition.tasklist_id }}/tasks
http_method: GET
error_handler:
type: CompositeErrorHandler
error_handlers:
- type: DefaultErrorHandler
max_retries: 5
response_filters:
- type: HttpResponseFilter
action: RATE_LIMITED
http_codes:
- 429
error_message: Rate limit hit
backoff_strategies:
- type: ExponentialBackoffStrategy
factor: 2
request_parameters:
maxResults: "{{ config['records_limit'] }}"
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path:
- items
partition_router:
type: SubstreamPartitionRouter
parent_stream_configs:
- type: ParentStreamConfig
stream:
$ref: "#/definitions/streams/tasks"
parent_key: id
partition_field: tasklist_id
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
$ref: "#/schemas/lists_tasks"
incremental_sync:
type: DatetimeBasedCursor
cursor_field: updated
end_datetime:
type: MinMaxDatetime
datetime: "{{ now_utc().strftime('%Y-%m-%dT%H:%M:%SZ') }}"
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
start_datetime:
type: MinMaxDatetime
datetime: '{{ config["start_date"] }}'
datetime_format: "%Y-%m-%dT%H:%M:%SZ"
datetime_format: "%Y-%m-%dT%H:%M:%S.%fZ"
cursor_datetime_formats:
- "%Y-%m-%dT%H:%M:%S.%fZ"
base_requester:
type: HttpRequester
url_base: https://tasks.googleapis.com/tasks/v1/
authenticator:
type: BearerAuthenticator
api_token: '{{ config["api_key"] }}'

streams:
- $ref: "#/definitions/streams/tasks"
- $ref: "#/definitions/streams/lists_tasks"

spec:
type: Spec
connection_specification:
type: object
$schema: http://json-schema.org/draft-07/schema#
required:
- api_key
- start_date
properties:
api_key:
type: string
order: 1
title: API Key
airbyte_secret: true
start_date:
type: string
order: 2
title: Start date
format: date-time
pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$
records_limit:
type: string
description: The maximum number of records to be returned per request
order: 0
title: Records Limit
default: "50"
additionalProperties: true

metadata:
assist:
docsUrl: https://developers.google.com/tasks/reference/rest
testedStreams:
tasks:
hasRecords: true
streamHash: 06d4c0949ab53aa3ea7e089781332fea60caf2f1
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
lists_tasks:
hasRecords: true
streamHash: 6d6137d9a88020f99badd5a98968cb1a12e07cdd
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
autoImportSchema:
tasks: true
lists_tasks: true

schemas:
tasks:
type: object
$schema: http://json-schema.org/schema#
required:
- id
- updated
properties:
id:
type: string
etag:
type:
- string
- "null"
kind:
type:
- string
- "null"
title:
type:
- string
- "null"
updated:
type: string
selfLink:
type:
- string
- "null"
additionalProperties: true
lists_tasks:
type: object
$schema: http://json-schema.org/schema#
required:
- id
- updated
properties:
id:
type: string
due:
type:
- string
- "null"
etag:
type:
- string
- "null"
kind:
type:
- string
- "null"
links:
type:
- array
- "null"
title:
type:
- string
- "null"
status:
type:
- string
- "null"
updated:
type: string
position:
type:
- string
- "null"
selfLink:
type:
- string
- "null"
completed:
type:
- string
- "null"
webViewLink:
type:
- string
- "null"
additionalProperties: true
Loading

0 comments on commit 37af2f0

Please sign in to comment.