Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remnux upgrade v2023.35.1 > v2023.36.1 Fails #150

Closed
ShanHolo opened this issue Sep 12, 2023 · 37 comments
Closed

Remnux upgrade v2023.35.1 > v2023.36.1 Fails #150

ShanHolo opened this issue Sep 12, 2023 · 37 comments

Comments

@ShanHolo
Copy link

Hello world

When I tried to upgrade to 2023.36.1 I had the following issues

Update returned exit code not zero
Error: Update returned exit code not zero
at ChildProcess. (/snapshot/remnux-cli/remnux-cli.js:571:23)
at ChildProcess.emit (events.js:315:20)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

Checking the saltstack log I figured out the following:

file_|-remnux-addon-version-file_|-/etc/remnux-version_|-managed:
id: remnux-addon-version-file
run_num: 780
sls: remnux.addon
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.remnux-python3-packages'
duration: 0.005
name: /etc/remnux-version
result: false

\ packages: pyelftools\n Attempting uninstall: pyelftools\n Found existing
\ installation: pyelftools 0.29 WARNING: Error parsing requirements for pyelftools:
\ [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/pyelftools-0.29.dist-info/METADATA'\n
\ WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n
ERROR: Cannot uninstall pyelftools 0.29, RECORD file not found. You might be
\ able to recover from this via: 'pip install --force-reinstall --no-deps pyelftools==0.29'."
duration: 2328.396
name: pyelftools
result: false

test_|-remnux-dedicated_|-remnux-dedicated_|-nop:
id: remnux-dedicated
run_num: 837
sls: remnux.dedicated
changes: {}
comment: 'One or more requisite failed: remnux.addon.remnux-addon-version-file'
duration: 0.004
name: remnux-dedicated
result: false

test_|-remnux-python3-packages_|-remnux-python3-packages_|-nop:
id: remnux-python3-packages
run_num: 413
sls: remnux.python3-packages
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.pyelftools.remnux-python3-packages-pyelftools'
duration: 0.033
name: remnux-python3-packages
result: false

Just in case I attach the saltstack log:

saltstack.log

@digitalsleuth
Copy link

Hi @ShanHolo , a new version was released just after you posted this issue which should resolve the problem you encountered. Can you try your install again?

@ShanHolo
Copy link
Author

ShanHolo commented Sep 13, 2023

Hi @digitalsleuth thanks for your quick response. I tried the upgrade again with the following errors:

pdate returned exit code not zero
Error: Update returned exit code not zero
at ChildProcess. (/snapshot/remnux-cli/remnux-cli.js:571:23)
at ChildProcess.emit (events.js:315:20)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

Errors on /var/cache/remnux/cli/v2023.37.1/saltstack.log

file_|-remnux-addon-version-file_|-/etc/remnux-version_|-managed:
id: remnux-addon-version-file
run_num: 784
sls: remnux.addon
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.remnux-python3-packages'
duration: 0.004
name: /etc/remnux-version
result: false

\ packages: pyelftools\n Attempting uninstall: pyelftools\n Found existing
\ installation: pyelftools 0.29 WARNING: Error parsing requirements for pyelftools:
\ [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/pyelftools-0.29.dist-info/METADATA'\n
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n
ERROR: Cannot uninstall pyelftools 0.29, RECORD file not found. You might be
\ able to recover from this via: 'pip install --force-reinstall --no-deps pyelftools==0.29'."
duration: 2620.682
name: pyelftools
result: false

test_|-remnux-dedicated_|-remnux-dedicated_|-nop:
id: remnux-dedicated
run_num: 841
sls: remnux.dedicated
changes: {}
comment: 'One or more requisite failed: remnux.addon.remnux-addon-version-file'
duration: 0.003
name: remnux-dedicated
result: false

test_|-remnux-python3-packages_|-remnux-python3-packages_|-nop:
id: remnux-python3-packages
run_num: 417
sls: remnux.python3-packages
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.pyelftools.remnux-python3-packages-pyelftools'
duration: 0.003
name: remnux-python3-packages
result: false

@geeksailor
Copy link

geeksailor commented Sep 13, 2023

Hi @ShanHolo!
I'll link my ticket but it has to do with pyelftools 0,29 folder existing in the dist-packages with no content, confusing salt. Try
sudo rm -rf /usr/local/lib/python3.8/dist-packages/pyelftools-029.dist-info to clear it out. Without it seeing a folder there it should work to try and run an update again.

#148

@digitalsleuth Is there a way in salt to test if the 0.29 folder exists already and if it does for salt to remove it prior to trying to update pyelftool? That might fix this issue for some.

@digitalsleuth
Copy link

Hi @ShanHolo , the release you attempted is missing one command, which is being pushed to the next release. It should be out shortly.
@geeksailor your ticket didn't have to do with pyelftools specifically, just that the pip/wheel/setuptools issue got hung up on that package vice others. That one should also be resolved with the upcoming release.

@ShanHolo
Copy link
Author

Thanks a lot for your appreciated support

Trying to update again to the version 37.2 I had the similar issue. @digitalsleuth did you mean the release 37.3? right

Thanks again.

@digitalsleuth
Copy link

@ShanHolo , yes 37.3 is the upcoming release, which hopefully we can get out today.

@geeksailor
Copy link

@digitalsleuth @ShanHolo Did the 37.3 do anything with Thug because I got a bunch of errors today relating to Thug and Speakeasy while trying to run the update. Otherwise I guess I might have done something to screw it up and the update process.

@digitalsleuth
Copy link

@geeksailor Can you provide your log file so I can see the errors? I know Thug released an update yesterday, but speakeasy hasn't had anything recently updated.

@geeksailor
Copy link

@digitalsleuth here's the grepped version. I can grab you the full stack too if you wish
saltstack_grepped_091523.txt

@geeksailor
Copy link

geeksailor commented Sep 15, 2023

Ran sudo pip3 show thug and got this spaghetti mess.
pip3_show_thug.txt
So I ran a sudo pip3 show setuptools since that seems to be the culprit and I got this
pip3_show_setuptools.txt

and a pip3 list for both of them for good measure

sudo pip3 list | grep setuptools
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
setuptools 68.1.2
setuptools-rust 1.7.0
remnux@ubuntu:/var/cache/remnux/cli/v2023.37.1$ sudo pip3 list | grep thug
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
thug 5.5

The Warning no metadata warning seems like the setuptools folder is broken and it is breaking everything else. Gonna go check it out
EDIT: Turns out I have 4 versions of Setuptools in my python3.8 dist-packages folder, 68.12, 68.2.0,68.2.1,and 68.2.2, an 68.1.2 only has the REQUESTED file in it and nothing else. Seems like it got removed for the most part minus that file and the folder and might be screwing up PIP when it does an inventory. Gonna try removing it and see what happens.

EDIT 2: Ok that got rid of the spaghetti of errors for doing the show on thug and setuptools. Gonna try running an update again and see if it works. Not sure what happened there to leave those scraps behind but hopefully it fixes it.

@geeksailor
Copy link

Side note, I am up to Remnux v2023.37.1 but I still see folders for 35.1 and 36.1 in my /var/cache/remnux/cli/v2023-3* directory. Should those have been auto cleaned up or is that something manual that I can remove?

@digitalsleuth
Copy link

@geeksailor Given that you've made a number of changes to your python setup (ie. manually deleting folders under /usr/local/lib/python3.8), this may be an issue with some of the changes you made. However, I will need the full saltstack.log file so I can see what occurred around the installation.

As for the over directories under /var/cache, you can delete 35.1 and 36.1, they don't get cleaned up because they retain the log files.

@geeksailor
Copy link

Ran another update and still have problems I think similar to the first. Haven't done a comparison yet.
Grepped version:
saltstack_grepped_091523_1311.txt
Full log:
saltstack.log

@digitalsleuth
Copy link

@geeksailor Can you try the following, exactly as below:
sudo python3 -m pip install 'setuptools=68.2.2' --force-reinstall

Once that is done, please take a look in the /usr/local/lib/python3.8/dist-packages/setuptools-68.2.2.dist-info/ folder and list off the items in there?

@geeksailor
Copy link

@digitalsleuth Will run it now. Be back in a minute with the results

@geeksailor
Copy link

geeksailor commented Sep 15, 2023

sudo python3 -m pip install 'setuptools==68.2.2' --force-reinstall
Collecting setuptools==68.2.2
Obtaining dependency information for setuptools==68.2.2 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
WARNING: Error parsing requirements for setuptools: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.2.0.dist-info/METADATA'
Installing collected packages: setuptools
Attempting uninstall: setuptools
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
Found existing installation: setuptools 68.2.0
ERROR: Cannot uninstall setuptools 68.2.0, RECORD file not found. You might be able to recover from this via: 'pip install --force-reinstall --no-deps setuptools==68.2.0'.

Seems I have another hangup folder. Removing and running again.

@digitalsleuth
Copy link

Try running the following now:
sudo touch /usr/local/lib/python3.8/dist-packages/setuptools-68.2.0.dist-info/METADATA
sudo touch /usr/local/lib/python3.8/dist-packages/setuptools-68.2.0.dist-info/RECORD

Do not remove the folder.

@digitalsleuth
Copy link

Once you've run those commands, try the previous command I gave you for 68.2.2

@geeksailor
Copy link

geeksailor commented Sep 15, 2023

Oh whoops I'll re-add it now. I think it is actually pip that is doing it and leaving these shells. It sees the old versions and tries to remove them but possibly doesn't clean it up completely

sudo python3 -m pip install 'setuptools==68.2.2' --force-reinstall
Collecting setuptools==68.2.2
Obtaining dependency information for setuptools==68.2.2 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 68.2.1
Uninstalling setuptools-68.2.1:
Successfully uninstalled setuptools-68.2.1

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
launchpadlib 1.10.13 requires testresources, which is not installed.
Successfully installed setuptools-68.2.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

ls setuptools-68.2.1.dist-info/
REQUESTED

@digitalsleuth
Copy link

Is the "REQUESTED" file the only thing in the folder?
Also, can you run sudo python3 -m pip list and provide that output?

@geeksailor
Copy link

geeksailor commented Sep 15, 2023

Yes, it is the only thing left.
sudo python3 -m pip list | grep setuptools

WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
setuptools 68.2.1
setuptools-rust 1.7.0

@geeksailor
Copy link

geeksailor commented Sep 15, 2023

ls | grep setup

setuptools
setuptools-68.2.1.dist-info
setuptools-68.2.2.dist-info
setuptools_rust
setuptools_rust-1.7.0.dist-info

ls setuptools-68.2.2.dist-info/

entry_points.txt INSTALLER LICENSE METADATA RECORD REQUESTED top_level.txt WHEEL

68.2.2 is also in there but not on the pip list?

@digitalsleuth
Copy link

Something seems to be wrong with the way pip is installing your packages. Try running this:
sudo python3 -m pip cache purge and then re-run the setuptools command I gave you previously.

@geeksailor
Copy link

geeksailor commented Sep 15, 2023

@digitalsleuth Sorry had some stuff that came up and had to respond to really quick.

sudo python3 -m pip cache purge

Files removed: 990

sudo python3 -m pip install --force-reinstall 'setuptools==68.2.2'

Collecting setuptools==68.2.2
Obtaining dependency information for setuptools==68.2.2 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
Downloading setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Downloading setuptools-68.2.2-py3-none-any.whl (807 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 807.9/807.9 kB 7.7 MB/s eta 0:00:00
WARNING: Error parsing requirements for setuptools: [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.2.1.dist-info/METADATA'
Installing collected packages: setuptools
Attempting uninstall: setuptools
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
Found existing installation: setuptools 68.2.1
ERROR: Cannot uninstall setuptools 68.2.1, RECORD file not found. You might be able to recover from this via: 'pip install --force-reinstall --no-deps setuptools==68.2.1'.

ls setuptools-68.2.1.dist-info/

REQUESTED

ls setuptools-68.2.2.dist-info/

entry_points.txt INSTALLER LICENSE METADATA RECORD REQUESTED top_level.txt WHEEL

sudo python3 -m pip list | grep setup

WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages
setuptools 68.2.1
setuptools-rust 1.7.0

sudo touch setuptools-68.2.1.dist-info/METADATA
sudo touch setuptools-68.2.1.dist-info/RECORD

sudo python3 -m pip install --force-reinstall 'setuptools==68.2.2'

Collecting setuptools==68.2.2
Obtaining dependency information for setuptools==68.2.2 from https://files.pythonhosted.org/packages/bb/26/7945080113158354380a12ce26873dd6c1ebd88d47f5bc24e2c5bb38c16a/setuptools-68.2.2-py3-none-any.whl.metadata
Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 68.2.1
Can't uninstall 'setuptools'. No files were found to uninstall.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
launchpadlib 1.10.13 requires testresources, which is not installed.
Successfully installed setuptools-68.2.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

I don't think touching those files helps. I can trying coping them from 68.2.2 to see if that works

@digitalsleuth
Copy link

Okay, after a bit of research and searching, it looks like this has become more widespread in the python community:
pypa/pip#12267
pypa/setuptools#3903

I'm going to need to rework the solution it seems.

@geeksailor
Copy link

Oomph. Sorry to add to the workload.

@digitalsleuth
Copy link

Not at all, I'm glad we're discovering this now so I can nip it in the bud for anyone else. Cheers!

@geeksailor
Copy link

Glad to contribute!

@ShanHolo
Copy link
Author

What a pity!! when I tried to update 37.3 version I had even more errors.

Even from a fresh Remnux VM installation I had the same errors.....

cat /var/cache/remnux/cli/v2023.37.3/saltstack.log | grep -i 'result: false' -B 8

file_|-remnux-addon-version-file_|-/etc/remnux-version_|-managed:
id: remnux-addon-version-file
run_num: 786
sls: remnux.addon
changes: {}
comment: 'One or more requisite failed: remnux.config.remnux-config, remnux.python3-packages.remnux-python3-packages'
duration: 0.009
name: /etc/remnux-version
result: false

id: remnux-config-bash-completion-unfurl
run_num: 681
sls: remnux.config.bash-completion
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.unfurl.remnux-python3-packages-unfurl-requirements,
remnux.python3-packages.unfurl.remnux-python3-packages-unfurl'
duration: 0.003
name: /etc/bash_completion.d/unfurl
result: false

file_|-remnux-config-bash-completion-xxxswf_|-/etc/bash_completion.d/xxxswf_|-managed:
id: remnux-config-bash-completion-xxxswf
run_num: 654
sls: remnux.config.bash-completion
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.xxxswf.remnux-python3-packages-xxxswf'
duration: 0.002
name: /etc/bash_completion.d/xxxswf
result: false

? file_|-remnux-config-volatility3-permissions_|-/usr/local/lib/python3.8/dist-packages/volatility3/framework/symbols_|-directory
: id: remnux-config-volatility3-permissions
run_num: 689
sls: remnux.config.volatility3
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.volatility3.remnux-python3-packages-volatility3'
duration: 0.007
name: /usr/local/lib/python3.8/dist-packages/volatility3/framework/symbols
result: false

file_|-remnux-python3-packages-dotnetfile-dump-shebang_|-/usr/local/bin/dotnetfile_dump.py_|-prepend:
id: remnux-python3-packages-dotnetfile-dump-shebang
run_num: 417
sls: remnux.python3-packages.dotnetfile
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.dotnetfile.remnux-python3-packages-dotnetfile-dump'
duration: 0.004
name: /usr/local/bin/dotnetfile_dump.py
result: false

file_|-remnux-python3-packages-dotnetfile-dump_|-/usr/local/bin/dotnetfile_dump.py_|-managed:
id: remnux-python3-packages-dotnetfile-dump
run_num: 416
sls: remnux.python3-packages.dotnetfile
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.dotnetfile.remnux-python3-packages-dotnetfile'
duration: 0.005
name: /usr/local/bin/dotnetfile_dump.py
result: false

file_|-remnux-python3-packages-old-speakeasy-wrapper_|-/usr/local/bin/run_speakeasy.py_|-absent:
id: remnux-python3-packages-old-speakeasy-wrapper
run_num: 390
sls: remnux.python3-packages.speakeasy
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy'
duration: 0.004
name: /usr/local/bin/run_speakeasy.py
result: false

file_|-remnux-python3-packages-speakeasy-emudll-shebang_|-/usr/local/bin/emu_dll.py_|-prepend:
id: remnux-python3-packages-speakeasy-emudll-shebang
run_num: 389
sls: remnux.python3-packages.speakeasy
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy-emuexe'
duration: 0.005
name: /usr/local/bin/emu_dll.py
result: false

file_|-remnux-python3-packages-speakeasy-emudll_|-/usr/local/bin/emu_dll.py_|-managed:
id: remnux-python3-packages-speakeasy-emudll
run_num: 388
sls: remnux.python3-packages.speakeasy
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy'
duration: 0.004
name: /usr/local/bin/emu_dll.py
result: false

file_|-remnux-python3-packages-speakeasy-emuexe-shebang_|-/usr/local/bin/emu_exe.py_|-prepend:
id: remnux-python3-packages-speakeasy-emuexe-shebang
run_num: 387
sls: remnux.python3-packages.speakeasy
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy-emuexe'
duration: 0.004
name: /usr/local/bin/emu_exe.py
result: false

file_|-remnux-python3-packages-speakeasy-emuexe_|-/usr/local/bin/emu_exe.py_|-managed:
id: remnux-python3-packages-speakeasy-emuexe
run_num: 386
sls: remnux.python3-packages.speakeasy
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy'
duration: 0.006
name: /usr/local/bin/emu_exe.py
result: false

file_|-remnux-python3-packages-volatility-rename-vol_|-/usr/local/bin/vol3_|-rename:
id: remnux-python3-packages-volatility-rename-vol
run_num: 351
sls: remnux.python3-packages.volatility3
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.volatility3.remnux-python3-packages-volatility3'
duration: 0.003
name: /usr/local/bin/vol3
result: false

file_|-remnux-python3-packages-volatility-rename-volshell_|-/usr/local/bin/volshell3_|-rename:
id: remnux-python3-packages-volatility-rename-volshell
run_num: 352
sls: remnux.python3-packages.volatility3
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.volatility3.remnux-python3-packages-volatility3'
duration: 0.004
name: /usr/local/bin/volshell3
result: false

   or: setup.py cmd --help\n      \n      error: invalid command 'egg_info'\n\
  \      [end of output]\n  \n  note: This error originates from a subprocess,\
  \ and is likely not a problem with pip.\nerror: metadata-generation-failed\n\
  \n× Encountered error while generating package metadata.\n╰─> See above for\
  \ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n\
  hint: See above for details."
duration: 2304.028
name: mwcp
result: false

? pip_|-remnux-pip3-stpyv8_|-/usr/local/src/remnux/stpyv8-ubuntu-20.04-3.8/stpyv8-10.1.124.12-cp38-cp38-linux_x86_64.whl_|-installed
: id: remnux-pip3-stpyv8
run_num: 339
sls: remnux.python3-packages.stpyv8
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.setuptools.remnux-python3-packages-setuptools'
duration: 0.003
name: /usr/local/src/remnux/stpyv8-ubuntu-20.04-3.8/stpyv8-10.1.124.12-cp38-cp38-linux_x86_64.whl
result: false

     or: setup.py cmd --help\n      \n      error: invalid command 'egg_info'\n\
[end of output]\n  \n  note: This error originates from a subprocess,\
  \ and is likely not a problem with pip.\nerror: metadata-generation-failed\n\
  \n× Encountered error while generating package metadata.\n╰─> See above for\
  \ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n\
  hint: See above for details."
duration: 2544.384
name: git+https://github.com/doomedraven/VirusTotalApi
result: false

or: setup.py cmd --help\n \n error: invalid command 'egg_info'\n
[end of output]\n \n note: This error originates from a subprocess,
\ and is likely not a problem with pip.\nerror: metadata-generation-failed\n
\n× Encountered error while generating package metadata.\n╰─> See above for
\ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n
hint: See above for details."
duration: 2907.461
name: git+https://github.com/pan-unit42/dotnetfile
result: false

or: setup.py cmd --help\n \n error: invalid command 'egg_info'\n
\ [end of output]\n \n note: This error originates from a subprocess,
\ and is likely not a problem with pip.\nerror: metadata-generation-failed\n
\n× Encountered error while generating package metadata.\n╰─> See above for
\ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n
hint: See above for details."
duration: 3414.335
name: git+https://github.com/TeamMsgExtractor/msg-extractor
result: false

or: setup.py cmd --help\n \n error: invalid command 'egg_info'\n
\ [end of output]\n \n note: This error originates from a subprocess,
\ and is likely not a problem with pip.\nerror: metadata-generation-failed\n
\n× Encountered error while generating package metadata.\n╰─> See above for
\ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n
hint: See above for details."
duration: 2552.826
name: git+https://github.com/buffer/ioc_parser
result: false

or: setup.py cmd --help\n \n error: invalid command 'egg_info'\n
\ [end of output]\n \n note: This error originates from a subprocess,
\ and is likely not a problem with pip.\nerror: metadata-generation-failed\n
\n× Encountered error while generating package metadata.\n╰─> See above for
\ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n
hint: See above for details."
duration: 2797.366
name: git+https://github.com/MITRECND/malchive.git
result: false

  \ file or directory: '/usr/local/lib/python3.8/dist-packages/pyelftools-0.29.dist-info/METADATA'\n\
  WARNING: Error parsing requirements for setuptools: [Errno 2] No such file or\
  \ directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.0.0.dist-info/METADATA'\n\
  \    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n\
  ERROR: Cannot uninstall pyelftools 0.29, RECORD file not found. You might be\
  \ able to recover from this via: 'pip install --force-reinstall --no-deps pyelftools==0.29'."
duration: 2621.571
name: pyelftools
result: false

  WARNING: Error parsing requirements for setuptools: [Errno 2] No such file or\
  \ directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.0.0.dist-info/METADATA'\n\
  \    WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n\
  ERROR: Cannot uninstall setuptools 68.0.0, RECORD file not found. You might\
  \ be able to recover from this via: 'pip install --force-reinstall --no-deps\
  \ setuptools==68.0.0'."
duration: 2165.034
name: setuptools==67.7.2
result: false

  (line 5)) (68.0.0)

  Requirement already satisfied: six>=1.11.0 in /usr/local/lib/python3.8/dist-packages
  (from jsonschema->-r https://raw.githubusercontent.com/mandiant/speakeasy/master/requirements.txt
  (line 5)) (1.16.0) ERROR: Could not install packages due to an OSError: [Errno
  2] No such file or directory: ''/usr/local/lib/python3.8/dist-packages/setuptools-68.0.0.dist-info/METADATA'''
duration: 960.481
name: remnux-python3-packages-speakeasy-requirements
result: false

pip_|-remnux-python3-packages-speakeasy_|-git+https://github.com/mandiant/speakeasy.git_|-installed:
id: remnux-python3-packages-speakeasy
run_num: 385
sls: remnux.python3-packages.speakeasy
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy-requirements'
duration: 0.004
name: git+https://github.com/mandiant/speakeasy.git
result: false

  \ --help-commands\n         or: setup.py cmd --help\n      \n      error: invalid\
  \ command 'egg_info'\n      [end of output]\n  \n  note: This error originates\
  \ from a subprocess, and is likely not a problem with pip.\nerror: metadata-generation-failed\n\
  \n× Encountered error while generating package metadata.\n╰─> See above for\
  \ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n\
  hint: See above for details."
duration: 7342.093
name: remnux-python3-packages-unfurl-requirements
result: false

pip_|-remnux-python3-packages-unfurl_|-git+https://github.com/obsidianforensics/unfurl.git_|-installed:
id: remnux-python3-packages-unfurl
run_num: 383
sls: remnux.python3-packages.unfurl
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.unfurl.remnux-python3-packages-unfurl-requirements'
duration: 0.004
name: git+https://github.com/obsidianforensics/unfurl.git
result: false

  \         or: setup.py cmd --help\n      \n      error: invalid command 'egg_info'\n\
  \      [end of output]\n  \n  note: This error originates from a subprocess,\
  \ and is likely not a problem with pip.\nerror: metadata-generation-failed\n\
  \n× Encountered error while generating package metadata.\n╰─> See above for\
  \ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n\
  hint: See above for details."
duration: 3436.878
name: git+https://github.com/volatilityfoundation/volatility3.git
result: false

  \         or: setup.py cmd --help\n      \n      error: invalid command 'egg_info'\n\
  \      [end of output]\n  \n  note: This error originates from a subprocess,\
  \ and is likely not a problem with pip.\nerror: metadata-generation-failed\n\
  \n× Encountered error while generating package metadata.\n╰─> See above for\
  \ output.\n\nnote: This is an issue with the package mentioned above, not pip.\n\
  hint: See above for details."
duration: 2449.317
name: git+https://github.com/viper-framework/xxxswf.git
result: false

pip_|-remnux-tools-peepdf-3-source_|-git+https://github.com/digitalsleuth/peepdf-3.git_|-installed:
id: remnux-tools-peepdf-3-source
run_num: 419
sls: remnux.python3-packages.peepdf-3
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.stpyv8.remnux-pip3-stpyv8'
duration: 0.004
name: git+https://github.com/digitalsleuth/peepdf-3.git
result: false

__id__: remnux-config
__run_num__: 690
__sls__: remnux.config
changes: {}
comment: 'One or more requisite failed: remnux.config.volatility3.remnux-config-volatility3-permissions,
  remnux.config.bash-completion.remnux-config-bash-completion-xxxswf, remnux.config.bash-completion.remnux-config-bash-completion-unfurl'
duration: 0.006
name: remnux-config
result: false

test_|-remnux-dedicated_|-remnux-dedicated_|-nop:
id: remnux-dedicated
run_num: 843
sls: remnux.dedicated
changes: {}
comment: 'One or more requisite failed: remnux.addon.remnux-addon-version-file'
duration: 0.004
name: remnux-dedicated
result: false

  remnux.python3-packages.ioc-parser.remnux-python3-packages-ioc-parser, remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy-emudll,
  remnux.python3-packages.volatility3.remnux-python3-packages-volatility-rename-volshell,
  remnux.python3-packages.xxxswf.remnux-python3-packages-xxxswf, remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy-requirements,
  remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy-emuexe-shebang,
  remnux.python3-packages.speakeasy.remnux-python3-packages-speakeasy-emudll-shebang,
  remnux.python3-packages.dotnetfile.remnux-python3-packages-dotnetfile-dump-shebang'
duration: 0.018
name: remnux-python3-packages
result: false

@digitalsleuth
Copy link

Hi @ShanHolo , to fix this, you can run the following then try the install again:
sudo python3 -m pip install --force-reinstall --break-system-packages "setuptools==68.0.0"

The reason for the setuptools 68.0.0 is because that is the broken package on your system:
Error parsing requirements for setuptools: [Errno 2] No such file or\ \ directory: '/usr/local/lib/python3.8/dist-packages/setuptools-68.0.0.dist-info/METADATA'\n\ \ WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages

The command above will fix the actual broken Python package, allowing it to be replaced during the next run.

@ShanHolo
Copy link
Author

Thanks a lot for you support!

However I still have errors.... I'm the only whit these errors trying to update?.

file_|-remnux-addon-version-file_|-/etc/remnux-version_|-managed:
id: remnux-addon-version-file
run_num: 786
sls: remnux.addon
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.remnux-python3-packages'
duration: 0.004
name: /etc/remnux-version
result: false

packages: pyelftools\n Attempting uninstall: pyelftools\n Found existing
\ installation: pyelftools 0.29 WARNING: Error parsing requirements for pyelftools:
\ [Errno 2] No such file or directory: '/usr/local/lib/python3.8/dist-packages/pyelftools-0.29.dist-info/METADATA'\n
\ WARNING: No metadata found in /usr/local/lib/python3.8/dist-packages\n
ERROR: Cannot uninstall pyelftools 0.29, RECORD file not found. You might be
\ able to recover from this via: 'pip install --force-reinstall --no-deps pyelftools==0.29'."
duration: 2348.943
name: pyelftools
result: false

test_|-remnux-dedicated_|-remnux-dedicated_|-nop:
id: remnux-dedicated
run_num: 843
sls: remnux.dedicated
changes: {}
comment: 'One or more requisite failed: remnux.addon.remnux-addon-version-file'
duration: 0.004
name: remnux-dedicated
result: false

test_|-remnux-python3-packages_|-remnux-python3-packages_|-nop:
id: remnux-python3-packages
run_num: 420
sls: remnux.python3-packages
changes: {}
comment: 'One or more requisite failed: remnux.python3-packages.pyelftools.remnux-python3-packages-pyelftools'
duration: 0.004
name: remnux-python3-packages
result: false

@geeksailor
Copy link

HI @ShanHolo!
It seems pyelftools left some fragments of 0.29.0 on it. Can you please do a sudo python3 -m pip list | grep pyelftool to see what version is installed?
Thank you!

@digitalsleuth
Copy link

Hi @ShanHolo , to fix this issue, you can run the following:

sudo python3 -m pip install --force-reinstall --break-system-packages pyelftools==0.29

This will fix the missing METADATA file and allow the remnux install to complete properly.
Let me know how it goes!

Cheers

@ShanHolo
Copy link
Author

Everything was solved!!!

I reinstalled pyelftools==0.29 and everything has been solved.

Thanks to @digitalsleuth and @geeksailor for your appreciated support. I learnt new things about the Remnux upgrade process.

@geeksailor
Copy link

@ShanHolo
Always glad to lend a helping hand! Glad it got resolved for you!

@digitalsleuth
Copy link

Good to hear @ShanHolo ! Cheers!

@digitalsleuth
Copy link

@lennyzeltser I believe we can close this one out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants