Pop Pulse News

shell wildcard expansion (un)safety


shell wildcard expansion (un)safety

Hi, Alexander Hu, CC'ed here, sent a message titled "shell expansion bug" to the distros list and a few other distro security contacts and shell maintainers. The message described known and correct behavior (not a bug), even if unexpected by some and risky. This was pointed out in the resulting thread. Since this was on the distros list, I am now bringing it to oss-security for transparency and consistency. My summary and additional thoughts are: Filenames may contain strings that are special to certain programs. Shells expand wildcards without care (nor reliable knowledge) of what programs the expanded results may be passed to, nor those programs' specific processing of any strings. The example given is of filename "--version", which ends up processed as an option by GNU grep. Since this issue and other related ones were known for decades, getopt(3) and getopt_long(3), which are used by many programs, will stop processing options upon seeing a plain "--" argument. It is to be used to separate options from further arguments (typically filenames). In other words, this is known unreliable/unsafe: grep text * whereas this is more reliable/safer: grep text -- * This avoids the issue described above, although it may not necessarily be exactly what's intended either - e.g., it omits filenames starting with a dot and it will fail when the number of filenames is too large. Processing arbitrary/untrusted directories reliably is tricky. For recursive processing, over the years we gained things like: grep -r text . where you don't need shell wildcard expansion, but instead pass a directory name, such as "." for the current directory, and even: find . -mindepth 1 -maxdepth 1 -type f -print0 | xargs -0 grep text -- where you can limit the recursion (or effectively disable it as in the example above), while also avoiding wildcard expansion (although you can still do wildcard matching with "-name"). The "-print0" and "-0" options deal with a related issue where filenames could contain linefeed characters, so we separate them by NULs instead, which filenames cannot contain (as NULs wouldn't get through POSIX's C string focused APIs). So there's no bug in the shells here, but bugs of this kind are common in shell usage (omitting the "--" argument where it's needed, usage of wildcard expansion where that is unnecessary, etc.) Can the shells do anything to mitigate this? I think not without breaking compatibility. The only not-too-unreasonable change I can think of is wildcard expansion prefixing filenames with "./", maybe only those that start with "-" and maybe not when used with builtin "echo". Since this didn't feel too unreasonable, I actually tested many shells to see whether any possibly already do this (with "/bin/echo *") - but I found none that do, at least not by default. I expect this would break e.g. explicit comparisons of expanded filenames for equality (to some expected filenames the script or another program checks for later) and uses of the filesystem as poor man's database (which scripts sometimes do). Maybe an opt-in mode or a future shell that doesn't need to stay 100% compatible with prior shells could do this, but even then I worry that it would encourage practices that would remain unsafe elsewhere. I'll include some excerpts from the distros list discussion below: On Thu, Oct 31, 2024 at 02:00:48PM +0100, Alexander Hu wrote:

Previous articleNext article

POPULAR CATEGORY

corporate

7781

tech

8841

entertainment

9702

research

4184

wellness

7536

athletics

9954