Integration tests

[dagwieers]

Check if this PR fulfills these requirements:

• My changes respect the Kodi add-on development rules
• I have read the CONTRIBUTING document.
• I made sure there wasn't another one Pull Requests opened for the same update/change
• I have successfully tested my changes locally

Types of changes

• New feature (non-breaking change which adds functionality)
• Feature change (non-breaking change which change behaviour of an existing functionality)
• Improvement (non-breaking change which improve functionality)
• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to change)

Description

This is the first set of integration tests. Please merge #194 first.

[dagwieers]

Running these:

run:
    @echo -e "$(white)=$(blue) Run CLI$(reset)"
    coverage run -a service.py &
    sleep 10
    coverage run -a test/run.py /action/purge_cache/
    coverage run -a test/run.py /action/purge_cache/?on_disk=True
    coverage run -a test/run.py /directory/root
    coverage run -a test/run.py /directory/profiles
    coverage run -a test/run.py /directory/home
    coverage run -a test/run.py /directory/video_list_sorted/myList/queue
    coverage run -a test/run.py /directory/video_list_sorted/newRelease/newRelease
    coverage run -a test/run.py /directory/video_list/continueWatching/continueWatching
    coverage run -a test/run.py /directory/video_list/chosenForYou/topTen
    coverage run -a test/run.py /directory/video_list/recentlyAdded/1592210
    coverage run -a test/run.py /directory/show/80057281/
    coverage run -a test/run.py /directory/show/80057281/season/80186799/
    coverage run -a test/run.py /directory/genres/tvshows/83/
    coverage run -a test/run.py /directory/genres/movies/34399/
    coverage run -a test/run.py /directory/search/search/cool
    coverage run -a test/run.py /directory/exported/exported
    pkill -ef service.py

Gives me a coverage of 48%:

[dag@moria plugin.video.netflix]$ coverage report --omit=test/*.py,modules/*,/usr/*
Name                                                   Stmts   Miss  Cover
--------------------------------------------------------------------------
addon.py                                                  71     35    51%
resources/__init__.py                                      0      0   100%
resources/lib/__init__.py                                  0      0   100%
resources/lib/api/__init__.py                              0      0   100%
resources/lib/api/data_types.py                          154     26    83%
resources/lib/api/exceptions.py                           24      0   100%
resources/lib/api/paths.py                                49     14    71%
resources/lib/api/shakti.py                              189     83    56%
resources/lib/api/website.py                             156    129    17%
resources/lib/cache.py                                   176     28    84%
resources/lib/common/__init__.py                          12      0   100%
resources/lib/common/cookies.py                           56     38    32%
resources/lib/common/credentials.py                       67     31    54%
resources/lib/common/data_conversion.py                   47      9    81%
resources/lib/common/fileops.py                           43     28    35%
resources/lib/common/ipc.py                               80     38    53%
resources/lib/common/kodiops.py                           85     59    31%
resources/lib/common/logging.py                           32     15    53%
resources/lib/common/misc_utils.py                       202    127    37%
resources/lib/common/pathops.py                           41     16    61%
resources/lib/common/storage.py                           42     27    36%
resources/lib/common/videoid.py                          168     50    70%
resources/lib/database/__init__.py                         0      0   100%
resources/lib/database/db_base.py                         21      7    67%
resources/lib/database/db_base_mysql.py                  137     89    35%
resources/lib/database/db_base_sqlite.py                 133     38    71%
resources/lib/database/db_create_mysql.py                 35     30    14%
resources/lib/database/db_create_sqlite.py                47     39    17%
resources/lib/database/db_exceptions.py                   12      0   100%
resources/lib/database/db_local.py                        59     22    63%
resources/lib/database/db_shared.py                      216    118    45%
resources/lib/database/db_update.py                       16     10    38%
resources/lib/database/db_utils.py                        50     25    50%
resources/lib/globals.py                                 175     41    77%
resources/lib/kodi/__init__.py                             0      0   100%
resources/lib/kodi/context_menu.py                        37      6    84%
resources/lib/kodi/infolabels.py                         117     36    69%
resources/lib/kodi/library.py                            331    251    24%
resources/lib/kodi/listings.py                           248     56    77%
resources/lib/kodi/nfo.py                                 96     74    23%
resources/lib/kodi/ui/__init__.py                          4      0   100%
resources/lib/kodi/ui/dialogs.py                          48     27    44%
resources/lib/kodi/ui/xmldialogs.py                       47     26    45%
resources/lib/navigation/__init__.py                      12      2    83%
resources/lib/navigation/actions.py                       66     39    41%
resources/lib/navigation/directory.py                    121     32    74%
resources/lib/navigation/hub.py                           13      6    54%
resources/lib/navigation/library.py                       69     44    36%
resources/lib/navigation/player.py                        91     65    29%
resources/lib/services/__init__.py                         7      0   100%
resources/lib/services/library_updater.py                 71     54    24%
resources/lib/services/msl/__init__.py                     0      0   100%
resources/lib/services/msl/base_crypto.py                 33     19    42%
resources/lib/services/msl/converter.py                  164    142    13%
resources/lib/services/msl/default_crypto.py              58     35    40%
resources/lib/services/msl/exceptions.py                  10      0   100%
resources/lib/services/msl/http_server.py                 43     26    40%
resources/lib/services/msl/msl_handler.py                168    121    28%
resources/lib/services/msl/profiles.py                    32      7    78%
resources/lib/services/msl/request_builder.py             43     21    51%
resources/lib/services/nfsession/__init__.py               0      0   100%
resources/lib/services/nfsession/http_server.py           27     14    48%
resources/lib/services/nfsession/nfsession.py            246    180    27%
resources/lib/services/playback/__init__.py                3      0   100%
resources/lib/services/playback/action_manager.py         37     19    49%
resources/lib/services/playback/controller.py             69     47    32%
resources/lib/services/playback/markers.py                22     14    36%
resources/lib/services/playback/resume_manager.py         20      9    55%
resources/lib/services/playback/section_skipping.py       51     33    35%
resources/lib/services/playback/stream_continuity.py      67     48    28%
resources/lib/services/playback/upnext.py                 18      8    56%
resources/lib/services/settings_monitor.py                53     41    23%
resources/lib/upgrade_actions.py                          35     24    31%
resources/lib/upgrade_controller.py                       40     25    38%
--------------------------------------------------------------------------
TOTAL                                                   5212   2723    48%

[CastagnaIT]

I also do login to codecov and added app to github but i do not understand where i have to put the codecov token?!?

You can set it in your CI environment variables. CODECOV_TOKEN....

[dagwieers]

@CastagnaIT You don't. In fact I think it would work out-of-the-box for codecov since this is a public repository.

[CastagnaIT]

this is ready to merge?

[dagwieers]

That is the question. Currently the integration tests fail in Travis, but work on my system.
For the integration tests to have any chance of working, you need to add 2 encrypted variables in Travis for this repo:

• NETFLIX_USERNAME
• NETFLIX_PASSWORD

This will ensure that the integration tests use your Netflix username and password.
(But for me this fails on Travis, maybe you have more luck?)

You can also set NETFLIX_ESN, and that seems to give a very specific error which I think is expected.

[dagwieers]

We could merge this PR, but then I would need to disable the integration tests from Travis. Otherwise it will look as if the build is broken.

[CastagnaIT]

any encrypted value is okay?

[dagwieers]

The reason we do not have any code coverage results is because we don't run any tests yet.
As long as we cannot make these integration tests work on Travis, Codecov has nothing to report :-/

So maybe we should merge this, without running it in Travis, and then we can look for a way to fix it in Travis. I may need your help with this.

[dagwieers]

You have to mark the variable so it is not shown in the logs.

[CastagnaIT]

i have added a fake username email and password and a real esn in travis Environment Variables, restarting this build seem nothing changed

[dagwieers]

Did you add it to Travis, because it requires a rebuild in that case.

And I don't see any evidence that you did in the current output.
Normally you would see this:

Setting environment variables from repository settings
$ export NETFLIX_USERNAME=[secure]
$ export NETFLIX_PASSWORD=[secure]
$ export NETFLIX_ESN=[secure]

[CastagnaIT]

i have clicked to restart job...it's wrong?

[dagwieers]

Oh right, it will not add them unless it is building the master branch. So it won't do it for PRs.
I guess that's because it is your repository so you cannot have your own PRs included.
(I can for the dagwieers account as it is my branch)

[dagwieers]

@CastagnaIT Maybe this can be merged while disabled in Travis. At least we can run it from the CLI.