Kodi support (former XBMC)

[synctext]

details to be determined in Jan 2017.

Video-on-demand, VLC, buffer management,..
Full keyboard control, efficient embedded operation, credit mining, smooth anon tunnels?

[synctext]

ToDo:

• Tribler basic docs (https://github.com/Tribler/tribler/wiki, Trusted peer discovery and improved NAT puncturing #2623 (comment))
• read Kodi docs
• try Bittorrent plug-ins
• develop HelloWorld plug-in for Kodi

[synctext]

Streaming related work, naughty plug-in: http://koditips.com/quasar-kodi-add-new-improved-pulsar-fork/

[EinNarr]

Progress Report:

Tribler Core is running externally. Quite some attempts have been done to make it run as a Kodi service add-on, but the reactor cannot be started properly. It is probably blocked due to mechanism Kodi is using. I am still seeking help on their forum, but I don't expect it can eventually be solved.

Currently channels can be properly discovered, subscribed and unsubscribed. Torrents can be found under a certain channel. But the interaction of subscribe/unsubscribe is a little bit inconvenient for the users, still trying to fix that. Search and download is not functional at them moment, expected to be implemented soon.

Current screenshots:

Next steps:

1. Enable configuring using Kodi interface.
2. Enable streaming using Kodi media player.
3. Try one last time to embed Tribler core.
4. Beautify the add-on, including the icons, thumbs and more hints.

[synctext]

For the thesis direction, please read :

• https://scholar.google.com/scholar?q=streaming+bittorrent&pagesize=100

[EinNarr]

Download monitoring and control done.
Play video locally down.
Configuration done.

Work left:

1. Streaming
2. Searching
3. Embed the core

[hbiyik]

if u upload code for addon, i can support, i am half way done my other kodi thingies

[hbiyik]

i am boogie btw

[synctext]

@hbiyik thnx for checking in! A code review would be really welcome. Search and downloading is now implemented.

@EinNarr can you please point us to your repo and push the latest code? (https://github.com/EinNarr/tribler/branches/all does not seem to be it)

[EinNarr]

@hbiyik Here is the repo: https://github.com/EinNarr/plugin.video.tribler
Thx for your help. A review wold be very helpful to me.

[hbiyik]

EinNarr/plugin.video.tribler#2

[synctext]

wow, that PR looks like 1 week of work! Was it?

Thank you very much, one of the bigger contribution we had for Tribler in 12 years since we started. You earned an official Tribler mug.. :-)

[hbiyik]

oh no, it looks huge because tribler core + dispersy and pure python dependencies are in it,
still it is a proof of concept code and needs lots of work however fixes several serious problems like:

importing precompiled .pyd modules to kodi python interpreter, uses PyInstaller distrbituon libraries under installed Tribler directory. (So in order make the addon work user must install tribler setup first)

Tribler Core python libraries (dispery, pymydht etc) to be submoduled to original branch. I used the version coming out of the installer from 7.0 alpha just to make sure it works, did not want to face complications coming from upstream. The good thing is Tribler Core is pythonically inside the addon, so tribler power is fully inside the addon.

There is a modified M2Crypto and cyprtography that loads compiled libraries from install dir under /lib dir. This should be distributed as is in plugin.video.tribler.

Also there are pythonic dependencies under /lib dir, these are supposed to be packaged seperately for
script.module.cherrypy
script.module.cheroot
script.module.tempora
script.module.configobj
script.module.decorator
script.module.enum
script.module.feedparser
script.module.portend

so in the end i could just work on tribler.py and service.py and addon.xml files only for 1 day.

But i still want that mug, it would be super cool to show off to my collages at work :).

I know there are bunch of things to work out but they all can be sorted out i think.

[EinNarr]

@hbiyik Awesome!
That PR really threatened me. +131,106 −1,145. xD
It would take quite some time for me to go through it.:)
Thank you so much.
BTW, is there somehow way to fully embed Tribler core into the add-on? Some users who hate tinkering would probably prefer a "one-click" button to fully install the add-on.

@synctext There is even a official Tribler mug?

[hbiyik]

@EinNarr
Just accept the PR, we will work on it more anyways, i think best way to start is first seperate those pure python libraries in to sperate kodi addons.

Meanwhile i am working on M2Crypto, cryptography, libnacl shared compiling and seperate addons. You can leave those nasty ones to me, have already complied for linux32 also gonna work on it.

yes at the end of the day addon will work standalone without trbiler installation. but those cbindings are quite nasty to work on cross platform.

I think better to start using issue tickets in your repo.

UPDATE: 05/04/17 Added more cbinded libraries to the list, and created a system that dynamically loads accroding to platform, architecture, and variants. Tested for windows and working like a charm. No more the need of Tribler setup. Currenly working on linuxes.

EinNarr/plugin.video.tribler#4

script.module.m2crypto=0.21.1
script.module.cryptography=1.2.1
script.module.libnacl=1.4.4
script.module.libtorrent=1.0.8
script.module.apsw=3.15.0
script.module.pywin32=220
script.module.plyvel=0.9
script.module.netifaces=0.10.5

[hbiyik]

https://github.com/hbiyik?tab=repositories

I have compiled cbinded libraries for manylinux-x86 (pretty much all of x86 32bit linux distros). and tested both on windows and linux. Now addon works independently with core and client.

I know you got little bit lost on it, if you have questions contact me on huseyinbiyik at hotmail.com via mail. We can chat online somewhere and discuss the details. It looks complicated at first but it is quite easy once you get the hang of it.

[EinNarr]

@synctext Is it possible to rearrange the meeting date to someday later this week? I got a presentation to do on Tuesday which was not on my calendar. Apologize for that.

[synctext]

sure, we reschedule.

[EinNarr]

@synctext What time shall that be? I should be fully available after tomorrow.

[EinNarr]

@synctext
I tested hbiyik's work on one of our Android phones.

1. It is very easy to replicate, simple copy and paste work, or maybe set up a Kodi repo to let the users install them in a even easier way.
2. In general, it works quite well, channels and torrents can be viewed in the same manner as on PCs. Downloads can be created, but I didn't wait to the end to see if it can be finally downloaded.
3. The phone is quite warm. It also happen with our generic Tribler android client so this should be just the feature of tribler core. We might need to reduce some functions when it is running on a phone.
4. Opening pages can be extremely slow, worse then the generic client. Might be improved by involving lazy loading.
5. Video server is not tested, due to the fact that it is hard to find an existing living piece of video (and is legal). I'd like to assume that it works.

[hbiyik]

@EinNarr

Opening pages can be extremely slow, worse then the generic client. Might be improved by involving lazy loading.

I think that is because i have run reactor inside a thread (i have to since python is itself a thread in kodi), and launch manycore may lead to work in single threaded in 2 core cpus. 1 core is the main thread, 1 core is the callbacks and deferreds, which will cause all deferreds to be blocking, this is my assumption of that issue didnt check in much detail, i will move to crochets for safer thread work, and im guessing that this is not related to tribler Core itself. Ill check in to that.
Crochets: https://crochet.readthedocs.io/en/1.4.0/
Related: https://github.com/hbiyik/script.module.triblercore/issues/3

[EinNarr]

@hbiyik I haven't tested that on PC. Would it also be slow? Maybe we can validate your assumption by disabling the extra cores on a PC.

Do you have any idea what is the best way to invoke tribler core? Currently I do a infinite loop at the start of the add-on until the core is up. But there will always be an error in the first run. Kinda no idea how to explain that.

[hbiyik]

@EinNarr
best way is ofc to create a session and utilize it to twisted loop, i fixed the startup problem with threads by dispatching xbmc monitor loop into twistedfromthread, i have fixed also the problem with shutting down so currently service.py handles the core loıop as it has to.

Also when refactoring the client part of it as we discussed on mail, i found out addon doing rest get config query each load up, this was partially a part of slow loading, other than that may be related to thread manager or lazy loading as you mentioned. I need to finish the refactoring client to prevent making a right from two wrongs. BTW it is not slow loading on PC but on my pc nothing is slow loading :) i am developing with an 8 core at 3200 ghz cpu :).

[EinNarr]

@hbiyik
As I mentioned in the e-mail, I originally thought that our user might have multiple Tribler client sharing a same account, thus the configuration need to be synchronized. However as our targeted users are probably using cellphones or TV boxes, this might not be that necessary.
Talking about performance, I heavily doubt if this could make a significant difference. For a quick test, you might comment out line 13 in plugin.video.tribler/resources/lib/module/setting.py after you run the add-on at least once on a device. It will only fetch the configuration from the xml file in Tribler add-on but not the core.

[EinNarr]

@hbiyik
Actually changing the policy to "only load configuration once" is pretty easy, just comment out that line from the init of the class, and invoke that method once right after the start of the core session.

[hbiyik]

@EinNarr
ok, usual suspects: threads. leveldb is depending boost on android builds, may be i have built boost with threads disabled, possible..
may be cpu specific optimizations are not done when cros compilning c modules possible but not likely, this generally happens on floating point calculations.
tribler core loop is blocked somehow, possible, but not likely, brusse's android client seems to be working fine.
or something very obvious that we havent noticed yet. so its a quality issue right now, and as long as addressed can be fixed anyways just need to dig in more.

[EinNarr]

@hbiyik

I tried to test it on a Google Pixel, but:

ImportError: Binary module is not compiled for android@ndk-x64_cp27m.

Seems that they are not using arm 7 any more

[synctext]

indeed the Snapdragon 821 in the pixel is v8:
https://en.m.wikipedia.org/wiki/Comparison_of_ARMv8-A_cores

[hbiyik]

@EinNarr yup its arm8 with 64bit architecture. Need to cross compile for those seperately. God thing abi detected atchiteture correctly :)
Qualcomm® Snapdragon™ 821
2.15Ghz + 1.6Ghz, 64Bit Quad-Core

[EinNarr]

@hbiyik I have to admit that you were right. Loading settings did slow the program down. I tested on Nexus 6 without loading settings from server and it is now visible faster. Yet, it is still quite slow, around 3s' loading time per page.

[EinNarr]

@hbiyik There is error when searching and sometimes changing settings in Android. Do you have the same problem?

[hbiyik]

@EinNarr, nice, i am pretty sure page loading problems at the end of the day is somehow related to some kodi specific issue, may be something is wrong during dispatching the page, exactly this is the reason why i am refactoring. However there seems to be another problem that slowing down the tribler core.

So after refactoring the code, page loading is fast (almost instant), i can confirm that in my fork, but some functionality may be slow still, so this will be another issue to handle on the testing phase.

Could not catch the issue with searches, currently i am working on custom UI creation which is arrrgggh tweaky, and yeah sorry i am little bit off lately, gf issues :).

[hbiyik]

@EinNarr small update: custom ui framework is done, now i have to edit channels and search endpoint for a healthy operation, :) seems like wherever i touch i need to modify to make it work. But i feel like things to do are "slowing down to expand", hopefully it will decrease, better slow than buggy.

[synctext]

epic progress..
Yes, this is the story of Tribler. After 12 years of progress, everything almost works..

[bam80]

Sorry.. What happened next? :)

[devos50]

As far as I'm aware, @hbiyik is currently working on this issue.

[hbiyik]

I want to update the current status with the Kodi Implementation

I have taken a complete different approach from the previous attempts, due to the facts that Tribler moved to Python3 and and Kodi is somehow is using python 2.6 to Python 3.x in the wild. So the current approach is, distributing the precompiled overall headless binary along with the addon.

There are certain challenges still:

• Streaming of tribler is not compatible with FFMpeg based custom streamer of Kodi: Status: PR Ready, may need further tuning. Streaming Improvements #5301
• Compiling the headless for various platfroms is prerequisite. For windows & linux I have done some trials and looks like it is not very challenging. The challenge is Android binaries. Maybe buildozer & P4A can be helpful this time.
• Kodi side addon development: This is least challenging part, I am doing the development in below repo for interested parties
  https://github.com/hbiyik/repository.biyik/tree/master/plugin.program.tribler

Here is a teaser of what we have already: A fully custom designed UI for Tribler ;)

[hbiyik]

related: #5327

[balupton]

A Plex plugin should be on the cards if a Kodi plugin is.

[synctext]

@hbiyik Still interested in Kodi support?
Tribler has been slow to develop in past 18 years and 4 months, but we slowly believing ourselves to have solved the Internet trust problem. Trust is what the Internet always needed, but never had... 🩹 🩸 🩹

[hbiyik]

@synctext
https://github.com/hbiyik/repository.biyik/tree/master/plugin.program.tribler

yeps, above is the initial work, latest commit 2 years ago, should work after few adaptations, the skeleton is there. I think i had given up at some point searching had some problems. If the api is stable now i can put more effort in it. Thanks for reminder.

But this issue might be closed if necessary i can create a new one whenever i encounter a pitfall

One problem i had was, having to compile the core binary for several targets by myself. Is there a pipeline for this in the CI?

Initially:
linux: armv7, armv8
windows: amd64, optionally x86

Future:
an apk for android.

For all above i have some previous work (for android quite experimental), if there is not something in tribler side thats also fine.

[hbiyik]

ah one last thing, i also remembered now, if kodi users are imported to Tribler i had a real concern that exit nodes in NL might actually overload. So how sure are you that the network is scalable at the moment?