Separate Connector from Core release lifecycle

[michel-tricot]

Tell us about the problem you're trying to solve

Today users have very little visibility when new versions of a connector are released. When we release core, we do so with a fixed set of connector versions. This isn't really a problem. The issue is that if we release a new version of a connector after the fact, the user has no good way (in the UI) to figure out that there is a new connector available.

They effectively get stuck on a connector version until they ask for help. We have numerous user issues due to connector version problems (e.g. issue). This is a common occurrence because connectors evolve at a different pace than core. It is pretty common that a user will not need upgrade their current core version but will want to upgrade a connector version to get a bug fix or new feature.

Describe the solution you’d like

I would like airbyte to be able to pull from a feed updates about connectors and detect:

• there is a new version and allow to update
• ensure that core is compatible with the new version of the connector. This should be less of an issue as we stabilize the protocol. (I propose we punt on this for now. It can be done on a follow on task with no additional overhead. The reason to punt is that it requires us to be more thoughtful about protocol version and we haven't figure that out yet, cgardens)

Describe the alternative you’ve considered or used

Today we have an escape hatch where if a user knows that there is a new version of a connector, they can manually input its version number in the UI. This process requires them to either read our change log or find where we declare new versions of connectors in the code base. This process is not ideal.

Issue for the UI work: #2218

[cgardens]

Spec that Davin is working on.