Base is not read-only

[3f6a]

julia> @edit LogExpFunctions.logaddexp(4, 5) # opened file is read-only (same happens for any other packages)

julia> @edit sin(4) # opened file is NOT read-only. Seems to occur for all base Julia installation files

Expected behavior: Base files should also be read-only. Seems permissions are not set correctly during installation?

I tested this on Mac (Apple silicon) and Linux (x86). In both cases I installed Julia via juliaup (just ran the script curl -fsSL https://install.julialang.org | sh).

julia> versioninfo()
Julia Version 1.9.3
Commit bed2cd540a1 (2023-08-24 14:43 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 8 × Apple M2
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, apple-m1)
  Threads: 1 on 4 virtual cores
Environment:
  JULIA_EDITOR = mvim
  JULIA_PKG_USE_CLI_GIT = true
  JULIA_PKG_SERVER_REGISTRY_PREFERENCE = eager

[longemen3000]

i can reproduce in windows, 1.9.1

[StefanKarpinski]

I suppose that installing Julia as read-only would make sense. Should we change this in the build process or the install process?

[stevengj]

Seems like this should be a packaging/installer thing?

[spaette]

julia 1.9.3

bash-5.1$ ls -l /usr/share/julia/base/math.jl
-rw-r--r-- 1 root root 39979 Aug 25 00:37 /usr/share/julia/base/math.jl
bash-5.1$ 

an ordinary (non-root) user opening math.jl

1. in emacs in the echo buffer appears the message "Note: file is write protected"
2. in vim it is listed as [readonly]

---

update

LogExpFunctions v0.3.26

bash-5.1$ pushd ~/.julia/packages/LogExpFunctions/W8b4e/src             
~/.julia/packages/LogExpFunctions/W8b4e/src ~
bash-5.1$ ls -l basicfuns.jl
-r--r--r-- 1 soeren soeren 11905 Aug 26 15:12 basicfuns.jl
bash-5.1$ popd
~
bash-5.1$ 

[KristofferC]

I think that is more based on how you installed Julia. A standard installation will not have that property.

[3f6a]

It's a juliaup issue then?

[KristofferC]

No, the same applies to the official installation method.

[spaette]

julia> @edit sin(4)

I presume @3f6a was running julia as an ordinary user.

julia-1.9.3-linux-x86_64.tar.gz

I suppose unless modified math.jl will have read/write perms for the ordinary or root user who extracted the precompiled .tar.gz.

Possibly over then next weeks on S.u.S.E. Leap I can look at juliaup.

I've the following currently on SuSE Linux (version 15.4) which runs KDE as the graphical desktop.

bash-4.4$ curl -s https://julialang-s3.julialang.org/bin/linux/x64/1.9/julia-1.9.2-linux-x86_64.tar.gz \
> | tar -xz
bash-4.4$ ln -s ~/julia-1.9.2/bin/julia ~/bin/julia
bash-4.4$ mkdir -p ~/.local/share/{applications,icons/hicolor/48x48/apps}
bash-4.4$ curl -s -o ~/.local/share/icons/hicolor/48x48/apps/julia.png \
> https://raw.githubusercontent.com/JuliaLang/julia/master/contrib/julia.png
bash-4.4$ cp ~/julia-1.9.2/share/applications/julia.desktop ~/.local/share/applications
bash-4.4$ gzip -9 ~/julia-1.9.2/share/man/man1/julia.1
bash-4.4$ ln -s ~/julia-1.9.2/share/man/man1/julia.1.gz ~/.local/man/man1/julia.1.gz
bash-4.4$ 

---

update

As a test I uninstalled the above and ran juliaup's curl -fsSL https://install.julialang.org | sh.

The math.jl file has the identical read/write perms as in the direct .tar.gz download as extracted.

bash-4.4$ pushd ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/base
~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/base ~
bash-4.4$ ls -l math.jl
-rw-r--r-- 1 soeren users 39979 Aug 24 10:36 math.jl
bash-4.4$ popd
~
bash-4.4$

Perms are the responsibility of the admin for a system installed program, or the user for a locally installed program.

[spaette]

read-only would make sense

packaging/installer thing

In the Un*x filesystem hierarchy standard /usr/share is for achitecture-independent data.

It's rather generic with Slackware Community submitted build scripts for source downloads to check perms.

SlackBuild Script Templates

$ ed -s autotools-template.SlackBuild <<<'110,114p'
find -L . \
 \( -perm 777 -o -perm 775 -o -perm 750 -o -perm 711 -o -perm 555 \
  -o -perm 511 \) -exec chmod 755 {} \; -o \
 \( -perm 666 -o -perm 664 -o -perm 640 -o -perm 600 -o -perm 444 \
  -o -perm 440 -o -perm 400 \) -exec chmod 644 {} \;
$ 

So one could add a shell script to precompiled julia downloads for non-root users to optionally run after local installs.

---

Of further note it's also rather generic to compress man pages.

$ ed -s autotools-template.SlackBuild <<<'145,150p'
# Compress man pages
# If the man pages are installed to /usr/share/man instead, you'll need to either
# add the --mandir=/usr/man flag to configure or move them manually after the
# make install process is run.
find $PKG/usr/man -type f -exec gzip -9 {} \;
for i in $( find $PKG/usr/man -type l ) ; do ln -s $( readlink $i ).gz $i.gz ; rm $i ; done
$ 

[3f6a]

I presume @3f6a was running julia as an ordinary user.

Yes, I am.

By the way, also the stdlibs are not read-only:

julia> using Random: bitrand
julia> @edit bitrand(4) #  file is not opened as read-only. Same for all stdlib source files

[spaette]

Binary Installation

Perhaps a command changing perms for locally installed pre-compiled binaries could be added in the above section.

Julia developers might determine if any tree output listed subdirectories should be excluded from such command.

bash-4.4$ pushd ~/julia-1.9.2/share 
~/julia-1.9.2/share ~
bash-4.4$ tree -L 1 --noreport julia
julia
├── base
├── base.cache
├── cert.pem
├── compiled
├── julia-config.jl
├── stdlib
└── test
bash-4.41$ popd
~
bash-4.4$ 

---

The juliaup code could be modified to determine whether local or system installs are being performed and change perms on the former.

[spaette]

@ViralBShah

The directory ~/julia*/share/julia/compiled is unaffected by running the command.

Marking PR a Draft.

This is the current command submitted on the premise that none of the affected files need be writeable.

if [ -d ~/julia*/share/julia ]; then
  find ~/julia*/share/julia/{base,stdlib,test} -perm 644 -exec chmod 444 {} \;
fi

This would be more comprehensive, again under the premise that none of the affected files need be writeable.

if [ -d ~/julia*/share/julia ]; then
  find ~/julia*/share/julia \
    -perm 755 -exec chmod 555 {} \+ -o -perm 644 -exec chmod 444 {} \+
fi

[spaette]

@davidanthoff

As a juliaup Collaborator any opinion on this ticket and pull request?

[spaette]

@longemen3000

can reproduce in windows, 1.9.1

JULIA_EDITOR

$JULIA_EDITOR takes precedence over $VISUAL, which in turn takes precedence over $EDITOR. If none of these environment variables is set, then the editor is taken to be open on Windows and OS X, or /etc/alternatives/editor if it exists, or emacs otherwise.

I think PowerShell accepts either forward or back slashes in the PATH.

(Get-ChildItem julia-1.9.3/share/julia/base/math.jl).Set_IsReadOnly($True)

The above PowerShell command can be adapted as per julia version and the location of the math.jl file.

FWIW, Notepad.exe does not indicate to the user when a file is readonly.

---

@andyleejordan

Download
julia-1.9.3-win64.zip

Would you and/or the PowerShell Team be amenable to recommending a command to recursively change the *.jl files to readonly?

The extracted *.zip file's etc directory tree contains the startup.jl file which can be left unchanged.

On Windows one would replace UnixMode with Mode in the below.

The Windows download unlike the Generic Linux download includes a julia-config.jl file that is not executable.

---

FWIW, this will list the *.jl files under the *.zip file's base directory tree.

Get-ChildItem -Path julia-1.9.3/share/julia/base -Recurse -Filter *.jl

[spaette]

PowerShell command

The below is authorised for use and/or modification by the julia and juliaup developers.

Windows (portable) [version neutral]

gci -Path $home\julia*\share\julia | gci -fi *.jl -r -File | % {$_.IsReadOnly=$True}  

Windows (installer) [version specific]

gci -Path $home\AppData\Local\Programs\Julia-1.9.3\share\julia *.jl -r -File | % {$_.IsReadOnly=$True}

Which file types to convert isn't my call; omitting the .jl in the commands should convert all file types to read-only.

I note in the share\julia subdirectory tree there are some .ci and .github directories and .gitignore and .*.yml files.

from @mklement0 [PowerShell repo Contributor]

Final thought: to make Get-ChildItem include hidden items too, add -Force.

a packaging/installer thing

https://julialang.org/downloads/platform/#windows

A checkbox could be added to the Windows installers, checked for ordinary user, unchecked for administrator.

Other:

☑ Add Julia to PATH
☑ Arch-independent data is read-only

[spaette]

I added PowerShell content to the pull; rebased and marked it ready for review.

Were the devs inclined to add the .sh and .ps1 scripts to the contrib directory this would be the README.md patch.

--- README.md.orig      2023-09-19 13:14:36.714260595 -0500
+++ README.md   2023-09-19 13:01:02.232205915 -0500
@@ -71,6 +71,8 @@
 [different tiers of support](https://julialang.org/downloads/#supported_platforms)
 for OS and platform combinations.
 
+It's recommended to correct the perms ([perms.sh](https://github.com/JuliaLang/julia/blob/master/contrib/perms.sh), [perms.ps1](https://github.com/JuliaLang/julia/blob/master/contrib/perms.ps1)) for locally extracted pre-compiled binaries.
+
 If everything works correctly, you will see a Julia banner and an
 interactive prompt into which you can enter expressions for
 evaluation.  You can read about [getting
@@ -111,6 +113,8 @@
 lists a series of running tests; if they complete without error, you
 should be in good shape to start using Julia.
 
+See the content from the previous section regarding perms.
+
 You can read about [getting
 started](https://docs.julialang.org/en/v1/manual/getting-started/)
 in the manual.

---

perms.sh

#!/bin/sh

# This file is a part of Julia. License is MIT: https://julialang.org/license

# run this script to correct perms for locally extracted pre-compiled binaries

# if extraction directory location is other than $HOME modify accordingly

# wget -q https://raw.githubusercontent.com/JuliaLang/julia/master/contrib/perms.sh
# ./perms.sh

if [ -d ~/julia*/share/julia ]; then
  chmod 444 ~/julia*/share/julia/{*.cache,*.pem}
  chmod 555 ~/julia*/share/julia/julia-config.jl
  find ~/julia*/share/julia/{base,stdlib,test} -perm 644 -exec chmod 444 {} \;
fi

perms.ps1

# This file is a part of Julia. License is MIT: https://julialang.org/license

# source this file to correct perms for locally extracted pre-compiled binaries

# if extraction directory location is other than $home modify accordingly

# iwr -Uri https://raw.githubusercontent.com/JuliaLang/julia/master/contrib/perms.ps1 -OutFile $home\perms.ps1
# . $home\perms.ps1

gci -Path $home\julia*\share\julia -ex compiled | gci -fi * -r -File | % {$_.IsReadOnly=$True}

[spaette]

Presuming root doesn't run the program, this ticket seems of concern for local and not system installs.

IMHO, the cleanest solution would be as follows.

A. overwrite the PR based on my previous comment #865
B. an added checkbox for Windows installers #865
C. juliaup devs implement their own solution

I guess macOS downloads are installed under /Applications and of consequence owned by root.

[stevengj]

I guess macOS downloads are installed under /Applications and of consequence owned by root.

No, MacOS downloads are owned by the user (with admin privileges) who installed the app (under /Applications or wherever):

% ls -l /Applications/Julia-1.9.app/Contents/Resources/julia/share/julia/base/math.jl
-rw-r--r--@ 1 stevenj  admin  39979 Jun  7 16:37 /Applications/Julia-1.9.app//Contents/Resources/julia/share/julia/base/math.jl

[spaette]

under /Applications or wherever

@stevengj

julia-1.9.3-macaarch64.dmg
julia-1.9.3-macaarch64.tar.gz

# if extraction directory location is other than $HOME modify accordingly

I guess it's open to interpretation whether macOS users would find the comment from perms.sh sufficiently clear.

Using the .dmg might not be construed as the equivalent of using a locally extracted pre-compiled binary.

A. overwrite the PR based on my previous comment #865
B. an added checkbox for Windows installers #865
C. juliaup devs implement their own solution

D. release .dmg downloads with corrected perms #865

[spaette]

I'd solicit an opinion on whether it's feasible to generate .dmg and .exe downloads as specified in this ticket.

B. an added checkbox for Windows installers #865

Installer checks if Admin or Non-Admin is running the .exe, and whether added checkbox is checked or unchecked.

D. release .dmg downloads with corrected perms #865

[spaette]

rewrote pull and rebased

@StefanKarpinski, @ViralBShah

please review pull in January

[StefanKarpinski]

I'm sorry, is there a pull request somewhere? This is an issue.

[spaette]

JuliaLang/julia#51150

The pull solely addresses locally extracted pre-compiled binaries.

[StefanKarpinski]

I'm a bit unsure of the premise that Julia should be installed read-only. I've been poking around my system and neither standard UNIX tools (like /usr/bin/*), nor applications (like /Applications/*) are installed read-only. Why should Julia specifically be read-only? If we decide that it should be installed read-only, then it seems like we should arrange for that to be done automatically rather than suggesting that people should run some script to fix permissions (approximately no one will do this). If we decide that it shouldn't be installed read-only, then it doesn't really make sense to tell people that they should run the script. We could offer the script as something that people can optionally run if they want to make Julia read-only, but then we should not say that it's recommended.

[stevengj]

One argument for installing the .jl files as read-only is that @edit Base.foo() will open up one of these files in an editor—this is a pretty common way to inspect the Julia source code, but it's generally not a good idea to modify the installed source. (I've done this by accident a few times myself, and I can easily imagine users thinking that this is a good way to patch Julia.)

[3f6a]

@StefanKarpinski I think you usually need admin rights to modify /usr/bin/* or /Applications/*, right? Whereas Julia is typically installed somewhere under ~/.julia/..., that a user can modify without being admin.

I agree with @stevengj's comment. This is the main reason why I opened the issue.

We only want that source files .jl of installed packages and Base to be read-only, not binaries or libraries, or other contents of .julia/. For instance, .julia/dev/, .julia/config/startup.jl, ..., should probably not be read-only. Source files of packages are already read-only.

So it would suffice to make the .jl source files of Base read-only.

[StefanKarpinski]

I think you usually need admin rights to modify /usr/bin/* or /Applications/*, right? Whereas Julia is typically installed somewhere under ~/.julia/..., that a user can modify without being admin.

In both places, programs are writeable by the owner of the program. In /usr/bin/ that's generally root so it's true that my user cannot modify them. In /Applications/ any app that I've installed is writeable by me.

I do, however, buy the argument that because we make it easy to open the installation's source files in an editor, we should prevent accidentally editing them. But in that case, I think we should not recommend a script, which approximately no one will run. Instead, we should arrange for the source files to be installed read-only.

[vtjnash]

It might be good to reland most of JuliaLang/julia#45441 then, with an update to make sure the files are read-only. That will finally fix the problem where out-of-tree builds install incorrectly and the build tree cannot be compatible with the install tree. But we potentially need an update to Revise.jl, to pick whether it wants to track the real (immutable) installed sources or the corresponding (writable) development files

[davidanthoff]

I assume files in tarballs can have a read-only flag? Juliaup just extracts the tar.gz downloads, so the way to implement this is presumably to just make sure the tar.gz release archives have the right read-only flags set.

[StefanKarpinski]

Files in tarballs can have any UNIX permissions (it was originally for tape archival purposes, after all). Our Tar.extract ignores everything but whether the user bit is executable but that doesn't matter much here since juliaup doesn't use that.

[spaette]

An end-user generally it is presumed likes things to work out-of-the-box.

I acknowledge non-root users running a post-install script or sourcing a post-install file may be a bit cumbersome.

Personally, I wouldn't want to have to add to tar xvf another switch when extracting the downloaded pre-compiled binaries.

The Unix Filesystem Hierarchy Standard (FHS) layout has /usr/share for architecture-independent (shared) data.

$ pushd ~/julia-1.9.2/share/julia/compiled/v1.9/DelimitedFiles
~/julia-1.9.2/share/julia/compiled/v1.9/DelimitedFiles ~
$ ls
dlKZm_psiZW.ji  dlKZm_psiZW.so  dlKZm_RmBmg.ji  dlKZm_RmBmg.so
$ find *  -print0 | xargs -0 file | grep -e "executable" -e "shared object" | grep ELF
dlKZm_psiZW.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
dlKZm_RmBmg.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
$ popd
~
$ 

Currently on Unix-like are files under share which are not arch-independent and such placement is construed as a deviation from that FHS.

FWIW, the .sh script modifies top level files separately from the selected subdirectories' files.

Julia is attributed in my fork as the owner of the .ps1 and .sh files and those files therefore can be modified by the developers.

[davidanthoff]

Yeah, I believe the tar extraction crate that is used by Juliaup does handle metadata, including readonly flags. So my best guess is that all we would need to do to make this work is set the readonly flag in the tarballs that Juliaup uses and we might just be done.

@spaette we're not proposing that end-users manually set any flags. The recommended way to install Julia these days is Juliaup, and it would take care of all of this for end-users.

[spaette]

recommended way to install Julia these days is Juliaup

#865

Personally, I tend to check perms irrespective of the Linux distribution in use.

#865

Julia developers might determine if any tree output listed subdirectories should be excluded from such command.

The last comment of @davidanthoff has been upvoted, and I'll leave it to a Maintainer to close my PR.

[3f6a]

@davidanthoff It would be nice if juliaup handled this automatically. Should this issue be transferred to Juliaup then?

[timholy]

But we potentially need an update to Revise.jl, to pick whether it wants to track the real (immutable) installed sources or the corresponding (writable) development files

I think Revise should track whatever was running. That is, if you want to edit the development source files, then you should be running a Julia built from those files. In that case I don't think Revise will need any modification.

Just chiming in here that I've been bit by this repeatedly (since juliaup's code is not a git repository, me and Ctrl-Z are getting to be quite good friends), so I'd be enthusiastic to see them made read-only.

[davidanthoff]

My understanding is that nothing needs to be done on the Juliaup side, but that the tar balls that Juliaup downloads just need to have the read-only flags set.

[StefanKarpinski]

I'm not clear what the hold up here is. Who needs do something? Do we need to change the way our tarballs are generated at build time? Does @staticfloat need to change how they're uploaded?

[davidanthoff]

I think we need to change how the tarballs are generated at build time, I believe the files that should be read-only should get that attribute inside the tarball, and then everything should/might work.

[spaette]

this ticket is closed

cf: #865 (comment)

my original command was as follows

find ~/julia*/share/julia/{base,stdlib,test} -perm 644 -exec chmod 444 {} \;

and with the commit's interpolated variables, I presume would have been written as

find $(DESTDIR)$(datarootdir)/julia/{base,stdlib,test} -perm 644 -exec chmod 444 {} \;