Which question and answer should be the canonical for all the many well-trodden duplicates/variants on the Python theme "Why is my command not being expanded by subprocess.run/call/Popen* / exec/ shlex/ etc.?", often asked as just "Why isn't my command working?" or "subprocess fails". Meanwhile, please start aggressively closing and stop answering new duplicates.
(The canonical answer is:
subprocess.Popen(..., shell=False)
is the default, you can either expand your wildcards with aglob
call, or you can enableshell=True
if you want, but it's a security hole when called with user input, so it's not recommended- But either way, whichever call you make to expand '*' wildcard, you then need to escape backslashes, and/or significant whitespace within file/directory names (note esp. Windows users). Otherwise that call will now start breaking for those new reasons.
- Also (duh!) check that the result of expanding the wildcards is non-empty, otherwise the final subprocess/ exec call will now fail for the reason that one of its filename/directory args is empty.
- Other things that
shell=True
provides other than wildcard expansion (excl. Windows), variable interpolation, and stdio redirection
Here are some of the many variant questions, but the canonical might well be something else, or maybe one of us just has to write one to cover all the many many variants (but a comprehensive answer will be very long, maybe too long to be readable. Thoughts?):
- Using a glob to generate arguments with subprocess.run()
- Wildcard not working in subprocess call using shlex
- Python subprocess Popen: Why does “ls *.txt” not work? [duplicate]
- Problems with command using * wildcard in subprocess
- subprocess popen argument with wildcard
- Subprocess command not finding files using ls command?
- python exec unix command - What is the error? [duplicate]
- Python's subprocess module returning different results from Unix shell
- https://stackoverflow.com/search?q=%5Bpython%5D+subprocess has 32.3K results
- https://stackoverflow.com/search?q=subprocess+glob gives 391 results
- https://stackoverflow.com/search?q=subprocess+wildcard has 55 results
- https://stackoverflow.com/search?q=%5Bpython%5D+popen has 9.4K results
- and truckloads more...
UPDATE:
- currently Linux only How does subprocess.call() work with shell=False?
shell=True
doesn't necessarily expand, depends on the shell (on windows, it doesn't)glob
expands. Others don't. your edit makes senseshell=True
is always a security hole, even when used with hardcoded commands. See the ShellShock attack which exploits the environment of the process to inject the command to run (by defaultsubprocess.Popen
& co inherit the parent environment). Luckily this should have been fixed but there surely are a ton of legacy system with this problem and there may be some other issue. So if you want to be safe when usingshell=True
you should always provide theenv
parameter too.shell=True
is for the lazy. It's good to test something/ensure that you can run the command in "dirty" mode, but there's always a solution to avoid it and it's more performant to go that way anyway because it doesn't run a shell, cannot redirect to files, whatever...