Open source the integrations server

[mvgorcum]

The integration server provides some of the core ideals of matrix: integrating other services and allowing easy bridging to other networks that are available.

As such a proprietary component used to provide this feels very much out of place in the otherwise open source matrix ecosystem. I get that new vector is looking at ways to monetize some aspects of matrix, which is perfectly fine, and writing some proprietary services or bridges makes sense in that context, but having riot for everyone on mobile and most on web to use a closed source solution seems quite contrary to the matrix philosophy.

This should also solve this issue that keeps riot labeled with nonfreenet on f-droid.

[edit]If this is in the wrong repository, please feel free to move, or ask me to move it.

[t3chguy]

@mvgorcum I'd recommend riot-meta: https://github.com/vector-im/riot-meta

[Half-Shot]

@t3chguy we had a chat about this oob, and I recommended web instead of meta because meta gets very little attention. To be honest, this question technically doesn't even fall under riot :|

[t3chguy]

I was mainly focussing on

but having riot for everyone on mobile and most on web to use a closed source solution

[mvgorcum]

I don't disagree that it may fit a bit better in riot-meta, but that's not a perfect place either. (It should obviously be in the scalar/modular integration server repo, but that one isn't open) Since riot-meta gets almost no attention I ended up deciding to open the issue here, lacking a 'perfect place' for this issue.

Still, though, if the consensus is that the issue doesn't belong here I'm fine with moving it elsewhere. Otherwise we may want to discuss the actual issue rather than its placement 😉

[lampholder]

I understand that the sentiment at the core of this is "having the shipped riot client coupled to a closed-source component seems antithetical to the general matrix philosophy".

There are few ways to tackle this: open sourcing the existing proprietary integrations manager would I feel only address the underlying concern in a superficial way - if we threw away scalar and switched to (open source) Dimension today, we'd still be shipping Riot clients (android, ios, riot.im/app) with a single hard-coded integrations manager instance.

The recent release of support for .well-known provides a neat way to address this coupling - by selecting the integration manager based on the homeserver's .well-known config, the homeserver administrator is able to direct which integration manager their users connect to.

Coming back to the question of open sourcing scalar - if shipped riot instances can be less closely coupled with an individual hard-coded integrations manager instance then whether or not to open source scalar becomes a purely commercial rather than a philosophical question - the integrations and integrations manager provided by New Vector are the only closed source components of the matrix ecosystem that we work on, and for the foreseeable future these will remain closed source.

[lampholder]

I recommended web instead of meta because meta gets very little attention

Yeah, tbh I use meta mostly for project planning and don't tend to triage issues that arrive there by any other means :\

[mvgorcum]

Thank you for the response.

There are few ways to tackle this: open sourcing the existing proprietary integrations manager would I feel only address the underlying concern in a superficial way - if we threw away scalar and switched to (open source) Dimension today, we'd still be shipping Riot clients (android, ios, riot.im/app) with a single hard-coded integrations manager instance.

I fully agree that the hard coding of the integrations manager is in itself already antithetical to matrix: it not being federated/decentralized in any way. Allowing the configuration of the manager through .well-known would be a good solution to this issue.

However: As it stands there is no other integrations manager available (be it open source or no) that works with the mobile clients. This means that .well-known in practice won't solve the issue for a large group of users. I suspect this is partially due to the fact that The Spec doesn't specify how the integrations API works, which should be fixed as well (though that should probably be discussed in matrix-doc).

Coming back to the question of open sourcing scalar - if shipped riot instances can be less closely coupled with an individual hard-coded integrations manager instance then whether or not to open source scalar becomes a purely commercial rather than a philosophical question - the integrations and integrations manager provided by New Vector are the only closed source components of the matrix ecosystem that we work on, and for the foreseeable future these will remain closed source.

For the foreseeable future it is highly likely that the default server for the vast majority of riot/matrix users is going to be matrix.org. As long as matrix.org defaults to the closed source solution, even .well-known won't solve this problem.

An alternative would probably be to give the user an explicit choice which integrations manager they want to use. As it stands that is chosen by whoever hosts riot, or through the distributed config.json on riot-desktop. Maybe a specific riot.im/app version allowing for a drop-down list to choose between scalar/modular and dimension?

I won't pretend I am not a little disappointed that scalar/modular won't be open sourced, but I can respect that choice. However, I would really like to see a bit more active work on getting at least an integrations manager that is open source working reliably for all riot clients. Specifically one that would also be accessible to the majority of general users (ie consumers that happen to use riot on matrix.org). Maybe the matrix.org foundation could sponsor travis to get the spec written and/or dimension compatible with riot mobile?

[turt2live]

Related: element-hq/element-web#6430, element-hq/element-web#4913, https://github.com/matrix-org/matrix-doc/issues/1286

[lampholder]

Tagging this wontfix just so it stops popping up in my list of untriaged issues :)

[poperigby]

Is there a discussion tag? That would probably better categorize this issue.

[onitake]

It seems there is a free alternative to the integration server available: https://github.com/turt2live/matrix-dimension
Would this help?

[frankwalter1301]

I understand that the sentiment at the core of this is "having the shipped riot client coupled to a closed-source component seems antithetical to the general matrix philosophy".

There are few ways to tackle this: open sourcing the existing proprietary integrations manager would I feel only address the underlying concern in a superficial way - if we threw away scalar and switched to (open source) Dimension today, we'd still be shipping Riot clients (android, ios, riot.im/app) with a single hard-coded integrations manager instance.

The recent release of support for .well-known provides a neat way to address this coupling - by selecting the integration manager based on the homeserver's .well-known config, the homeserver administrator is able to direct which integration manager their users connect to.

Coming back to the question of open sourcing scalar - if shipped riot instances can be less closely coupled with an individual hard-coded integrations manager instance then whether or not to open source scalar becomes a purely commercial rather than a philosophical question - the integrations and integrations manager provided by New Vector are the only closed source components of the matrix ecosystem that we work on, and for the foreseeable future these will remain closed source.

The use of integration servers provided by the homeserver is now supported. I guess it makes sense now to make open source scalar.

[pbek]

Is there a way to report issues to the integration server, e.g. element-hq/element-web#19355 (comment)?

[turt2live]

If it's directly related to the integration manager then the issues are opened on this repo. Looks like the instruction from that issue is to open the issue on a different repo.

Please use #element-web:matrix.org on matrix for further queries.

[pbek]

Ok, thank you. I already opened element-hq/element-web#19355 (comment) (which was closed).

[jittygitty]

Has this been addressed? I was shocked to hear the "integrations" server which appears under "Integration manager" as "scalar.vector.im" is not even open sourced. Maybe that's not a deal breaker but how do we change it to a different server?

Its pretty crazy considering the way element promotes itself for 'privacy'/encryption etc, to then force usage of your servers if we want to use any integration bridges, widgets, bots or even sticker packs, this seems like one big purposeful analytics leak?

Because I just found https://github.com/turt2live/matrix-dimension and there might be others?

matrix-org/synapse#7702

[turt2live]

The status on this has not changed: you can run your own integrations manager no problem. The team does not wish to open source Scalar though, and is in fact looking at ways to decommission it in favour of a more reliable/maintainable option.

You are not forced to use Scalar. Please consult the documentation for the client of your choice for more information.

I'm going to lock this as the concern has been addressed as of years ago. If people have further questions, please visit #element-web:matrix.org on Matrix.

[richvdh]

Seems like this is no longer relevant? Can it be closed?

[Half-Shot]

Yes, believe so. #86 (comment) still stands, though with a few new updates:

• Scalar has been fully decommissioned and is no longer relevant.
• The new integration manager will not be open sourced but previously closed source components of Scalar have open source replacements. The Hookshot, IRC and Slack bridges are now providing their own interfaces as part of the applications, which is fully open source. The integration manager now only provides a wrapper around those.

[progval]

The new integration manager will not be open sourced

Why?

[tengkuizdihar]

Bumping up this issue again, may we get a list of closed sourced component that's used in the matrix ecosystem by your company beside this integration manager @Half-Shot?

I'm using and supporting the use of matrix because of its open source nature, it would be weird if one of the nice things about it somehow implemented in closed sourced way.

[ara4n]

Element's integration manager happens to be proprietary, and always has been since it was created in 2017 or so.

Meanwhile, open source integration managers also exist, like https://dimension.t2bot.io/ - and folks are very welcome to write their own. (It's not even that hard, as Dimension shows).

may we get a list of closed sourced component that's used in the matrix ecosystem by your company beside this integration manager

Element has a bunch of proprietary products, which we use to try to fund our salaries to keep working on Matrix. https://element.io/server-suite has a list of them.

Just because Element happens to sell some proprietary products does NOT mean that Matrix itself is in any way proprietary; it's just that being an open standard, people can choose whatever license they like to build on top. For instance, Beeper is a proprietary Matrix client. Element is a FOSS Matrix client. The Integration Manager is a proprietary component.

Element has no plans to FOSS the integration manager right now - we are continuing to fight to get to break-even (i.e. stop losing money), and we need everything we can to do so.

[t3chguy]

Meanwhile, open source integration managers also exist, like dimension.t2bot.io

Worth noting Dimension is no longer maintained and has been archived https://github.com/turt2live/matrix-dimension#-project-archived