PRoot: User-space implementation of chroot, mount –bind, and binfmt_misc

132 points by nateb2022 7 days ago | 50 comments

nine_k 7 days ago |
Ah! Finally I see a way to try nix (the package manager) on my existing installation without it requiring access to the root directory and other such stuff!
yjftsjthsd-h 7 days ago |
https://nixos.wiki/wiki/Nix_Installation_Guide discusses that a bit more (also other solutions)
lostmsu 7 days ago |
Unrelated, but does anyone know what's the deal between wiki.nixos.org and nixos.wiki? The former appears to be actually functional, but the latter ranks much higher in search engines and seems to be incorrectly treated by them as the primary source.
Reventlov 7 days ago |
Because nixos.wiki was the primary source, until very recently. Then, wiki.nixos.org was introduced, and mass edits were made (without the owner of nixos.wiki being ok with that) trying to redirect people from nixos.wiki to wiki.nixos.org, and now both co-exist.
Reminds me a bit the archlinux.fr vs wiki.archlinux.org fr wiki situation from 10 years ago.
Voklen 7 days ago |
nixos.wiki was the primary wiki before NixOS really had an official one. Now there is the official wiki.nixos.org and the NixOS foundation asked the nixos.wiki maintainer to cooperate on the new wiki but they refused. Now there are multiple problems[1] with nixos.wiki and the maintainer in unresponsive so although unfortunately it's ranked higher in search results, in my experience you'll get better information on the official wiki.
[1] https://wiki.nixos.org/wiki/FAQ#Why_is_there_a_new_wiki?_Wha...?
lostmsu 6 days ago |
Ah, too bad the maintainer is unresponsive. If they don't plan to work on it, they'd do a huge service to community by adding permanent redirects to wiki.nixos.org and maybe canonical links.
Y_Y 6 days ago |
I went to see what the problems were for myself. While I wouldn't run my own sure that way, I was expecting something more serious to justify a migration. I think having the primary wiki within the project is a good idea generally. It does feel like the problems below are just minor annoyances though.
---
Why is there a new wiki? What is with nixos.wiki?
The old wiki at nixos.wiki has several problems:
Many components (mediawiki, php, icu) are severely outdated. MediaWiki 1.29 (EOL 2018), now 1.35 (EOL 2023-12) PHP 7.3.33 (EOL 2021-12) ICU 64.2 Cloudflare DDOS protection makes wiki edits fail sometimes. There is no WYSIWYG editor. The wiki infrastructure, which was supposed to be made public after launch, never ended-up being made public
yjftsjthsd-h 6 days ago |
Many components (mediawiki, php, icu) are severely outdated. The wiki infrastructure, which was supposed to be made public after launch, never ended-up being made public
I suspect both of these are a bigger deal to the Nix community than perhaps to the average group.
kokada 7 days ago |
I don't recommend using Nix inside PRoot unless this is your only option (e.g.: Android, since there are no user namespaces). The reason for that is that Nix is syscall heavy and using it in PRoot will be slow, really slow.
As someone said, the wiki [1] has some interesting options. From the options I used nix-user-chroot [2] with great success, and while the tool is unmaintained it should still work. It uses Linux's user namespaces instead of intercepting syscalls, and this makes the performance pretty much identical to native.
If you want more control like PRoot offers, I recommend bubblewrap instead [2].
Another option undocumented is to just grab a recent nix binary somewhere and run. It will automatically create the Nix store inside `$HOME/.local/share/nix` (if I am not mistaken) and use user namespaces to mount to it. However the last time I tried this didn't work well for a few things (e.g.: Home-Manager), so in general I still think nix-user-chroot is a better choice.
[1]: https://wiki.nixos.org/wiki/Nix_Installation_Guide, linking to the official one
[2]: https://github.com/containers/bubblewrap
[3]: https://wiki.nixos.org/wiki/Nix_Installation_Guide#nix-user-...
soraminazuki 6 days ago |
Nix can do that natively with the --store option. But I'd recommend installing it normally if you do have access to / on your system though. Nix itself doesn't litter files all over the filesystem and can be uninstalled cleanly.
ggm 7 days ago |
Unless I am mistaken this cannot elide over the fundamental protections Android enforce which make it impossible to do cross-device symlinks to external SD cards in a non-rooted Android: the outer kernel blocks this, even though you can cd into the paths.
If (for example) you can overlay/union mount and have a synthetic upper layer FS which manages things, I could use this.
hks0 7 days ago |
proot works by hijacking and altering syscalls. So maybe you could emulate the symlink yourself, by altering the file paths at ptrace level?
Thinking more, maybe it works out of the box? Just mount the overlays (there's a cli flag IIRC) and proot takes care of syscalls by definition.
CaliforniaKarl 7 days ago |
Here's an example of how we've used this.
RStudio Server[0] 1.3 and older hard-coded a number of paths, such as the path for storing temporary files: Instead of looking for the TMPDIR environment variable (as specified by POSIX[1]), R Studio Server would always use /tmp. That is extremely annoying, because we set TMPDIR to a path on fast local storage (SATA or NVMe SSDs) that the job scheduler cleans up at the end of the compute job.
We do have a last-resort mechanism using pam_namespace[2], such that a user going to `/tmp` actually takes them to `/namespace/tmp/${username}`, but that is per-user, not per-job. If a user has two R Studio jobs, and those two jobs landed on the same host, there would be trouble.
So, we used PRoot to wrap R Studio, with /tmp bind-mounted to a directory under TMPDIR.
[0]: https://www.rstudio.com/products/rstudio/download-server/
[1]: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1...
[2]: https://linux.die.net/man/8/pam_namespace
godelski 7 days ago |
For those that don’t know, you shouldn’t blindly let programs have access to tmp. They can get access to sockets and stuff. If you’re running with systemd there’s a private tmp option for this reason.
It’s always best to sandbox programs when you can. Linux has been making this much easier but it’s still non trivial
https://salvatoresecurity.com/the-many-perils-of-tmp/
westurner 7 days ago |
From the systemd.exec man page: https://www.freedesktop.org/software/systemd/man/systemd.exe... :
PrivateTmp=true #JoinsNamespaceOf=
`unshare -m` and then bind-mounting a private /tmp at /tmp/systemd-private-/ does the same thing; `systemd-tmpfiles --help`: https://serverfault.com/questions/1010339/how-exactly-to-use...
mikepurvis 6 days ago |
"`unshare -m` and then bind-mounting a private /tmp at /tmp/systemd-private-/ does the same thing"
As an intermediate level user and sysadmin, this kind of thing underscores the good work systemd is doing making it easy to get sane and safe defaults for things otherwise fiddly enough that many normal people wouldn't bother.
westurner 6 days ago |
And in a config file format so we don't have to add things like renice, respawn, chroot, and namespaces to a locally forked Sys-V unit script for just one distro.
Thankful for systemd too.
mjevans 6 days ago |
Isn't that the entire point of each socket living inside of a directory with privs / ACL protecting it?
godelski 5 days ago |
Yeah that helps but it depends on the permission of the socket. You can make it harder to guess by changing control path.
Here’s an article that talks about tmp[0]. But I mean if you wanted to be really secure you could use systemd nspawn and put each user into their own container[1]
[0] https://salvatoresecurity.com/the-many-perils-of-tmp/
[1] https://kilabit.info/journal/2022/chrooting_ssh_user_into_sy...
paulyy_y 6 days ago |
Please don't use zero-based footnote citations, people will assume 1 is the first one (because that is the standard in like, all of literature) and muscle-memory click the first link when reading and have to then figure out that you are doing a non-standard thing.
yjftsjthsd-h 6 days ago |
I disagree; this is Hacker News, and the cultural background favors zero indexing.
JonChesterfield 6 days ago |
The natural numbers start from zero.
zamadatix 6 days ago |
Funnily enough, that's exactly the debate (whether or not natural numbers start at 0 or 1). There is no universally right or wrong answer, is comp sci nerds just tend to like 0 :p.
Y_Y 6 days ago |
Everyone knows N_0 == N\{0} == the natural numbers without zero. This resolves the ambiguity and was my go-to for zero-denier professors in undergrad.
> But the natural numbers are for counting, you can't count to zero!
Why don't you count the number of cool people who don't think zero is a natural number, loser?
[0] also it works beautifully for pointer arithmetic
malkia 6 days ago |
This ain't Lua, Pascal, Basic or Real Life :)
godelski 6 days ago |
Looks like your new here, let me help out.
Language is not defined by a dictionary or MLA guides or whatever. Language is defined by culture. Go ask any linguist if your in doubt. They’ll gladly talk your ear off about how language evolves and cliques form and how there’s in and out language which people can often identify others in their “tribe” by. Like respect for any culture, don’t come in and try to tell people what’s right or wrong. At least not until you can speak the language first.
We use [0] because it’s how you index an array.
More general to internet culture there’s also OP = original poster and we refer to comments above us like family relationships. Parent (you’re this comments parent), GP (the grandparent, one who used the [0] that triggered you), GGP, G..GP, there’s siblings/sister (everyone who replied to you) and so on. Generally people don’t say cousins, aunts, dust removed and so on but you might see it). RTFM = read the fucking Manual (RTFA for article). And many others.
Welcome to the internet and Hacker News.
hks0 7 days ago |
Termux has now a nice wrapper (manager?) for proot [1]. I once wanted to submit a wrapper but got an angry response as the first comment to my PR and it was closed. Guess enough people did that until they decided maybe it's not a bad idea after all.
[1]: https://github.com/termux/proot-distro
yjftsjthsd-h 7 days ago |
> I once wanted to submit a wrapper but got an angry response as the first comment to my PR and it was closed.
I don't follow; you made a PR to add this functionality to proot itself?
hks0 7 days ago |
Ah no, sorry I was so lost in my thoughts and explained badly.
I was taking in the context of Termux app for Android, nothing to do with proot itself. Termux uses apt and deb repos, and provides a patched proot. I wanted to add a package similar to one I linked.
jillyboel 6 days ago |
> I once wanted to submit a wrapper but got an angry response
link?
albertzeyer 7 days ago |
A similar tool for this user-space bind-mount is https://github.com/fritzw/ld-preload-open, which relies on LD_PRELOAD to overwrite common libc functions. Thus this is less reliable as the presented tool which uses ptrace, but it still works reasonably well (I run e.g. PyCharm with it).
jeduardo 7 days ago |
Thanks for sharing this! I had to do exactly the same thing some 10 years ago to get an Oracle instance up and running again. Oracle insisted on using the /tmp location, despite being installed on a different drive, and the disk was full. As I had access to the Oracle system user, but not to the DBA user to change any configuration, I built a similar shared lib and preloaded it to the script. Worked like a charm! Happy to know that there is something _slightly more streamlined_ to do that now.
AshamedCaptain 7 days ago |
There is also Scratchbox, which was used (and probably is still used) to cross-build embedded Linux distributions. https://github.com/sailfishos/scratchbox2
It also offers CPU transparency and was able to run almost arbitrary desktop software, but specializes in build toolchains.
aloisdg 7 days ago |
Great name in french
QuiDortDine 6 days ago |
Means "fart" for the curious.
Voklen 7 days ago |
Can anyone explain why chroot requires root privileges in the first place? Because from my understanding it seems like it should only restrict what you can do rather than grant any new abilities.
badmintonbaseba 7 days ago |
Nowadays you have unprivileged options with mount namespaces. And from an unprivileged user namespace you can probably invoke chroot just fine as "root".
`chroot`, the function, has probably some baggage that makes it impossible to enable straight from an unprivileged program? Anyway, a chroot-like program seems to be implementable on top of user namespaces. Rootless docker is mostly that, with more namespace and cgroup isolation.
Teongot 7 days ago |
If you can get a suid root binary into the chroot, then you can control its configuration files to bypass security restrictions.
$ ln /usr/bin/sudo ./my-chroot $ echo "$USER ALL=(ALL) NOPASSWD: ALL" > ./my-chroot/etc/sudoers.d/01-oops $ chroot ./my-chroot $ sudo bash
modern Linux distributions prevent creation of hard links to suid binaries, but the restrictions on chroot came years before that.
cedws 7 days ago |
Can’t SUID binaries in chroots be ignored, like the nosuid mount flag?
blueflow 6 days ago |
Linux namespaces can do that, with a UID namespace, you drop the ability to do SUID, and then you can open a mount namespace and bind-mount and chroot as you like.
vifon 7 days ago |
chroot doesn't nest, there is only one chroot active for a given process at a given moment. If you're inside a chrooted environment and call chroot on a subdirectory without entering it, you regain the access to the parent directories.
compsciphd 6 days ago |
which is why before fs namespaces, as part of security research prototype work I did, I created a "chroot aware" (not quite, but close enough for this context to use that term), that wouldn't let one walk past certain directories (i.e. lookup() would fail for them, no matter what permissions the user had once one entered "pseudo namespace" mode via this chroot mechanism). Was very easy to accomplish, but also very much a hack that fs namespaces are much better for.
rekado 7 days ago |
`guix pack` can create bundles that use a static proot to make them relocatable:
https://hpc.guix.info/blog/2017/10/using-guix-without-being-...
It also supports other more performant ways, but in some situations proot is the best choice.
Y_Y 6 days ago |
But can it pack the whole system? I've been trying to run Guix on my Android for a while without resorting to a full VM. Nix has a custom termux, and lots of distros run under proot under termux[0], but seemingly not Guix.
[0] https://wiki.termux.com/wiki/PRoot
randomtoast 7 days ago |
If I shell script something together that executes a process in a PRoot environment, would that be similar to Docker?
badmintonbaseba 7 days ago |
So this works by intercepting file-related syscalls. Does it intercept io_uring?
bieganski 7 days ago |
related url, for syscall intercepting made easy: https://github.com/bieganski/asstrace/
see `pathsubst` example.
unfortunately set of use cases for `ptrace`-based solution is limited, due to high performance overhead.
animesh10k 6 days ago |
this is epic stuff
molticrystal 6 days ago |
Does this support using gdb to debug a program, because I vaguely remember years ago using a simulated root to install a linux distro on android to compile and test certain ideas while traveling and due to the nature of ptrace , they couldn't be loaded into a debugger.
gbraad 6 days ago |
have been using proot on Android to start my container image[1] as an alternative OS on top of termux. it is not perfect, but at least you can have a Fedora userspace on an Android tablet. when you add termux:x11 or vnc you have a minimal desktop, and with vscode a dev environment.
1: https://github.com/gbraad-devenv/