Bug #6029

A solution for the problem of shell=True (Python module subprocess) in Rutils.py on Mac OS X and other Unix system

Added by Martin Laloux almost 2 years ago. Updated over 1 year ago.

Status:Closed Start Date:07/12/2012
Priority:High Due date:
Assigned to:Victor Olaya % Done:

0%

Category:Processing-Backend/R
Target version:-
Patch supplied:Yes Tag:

Description

I could not use R commands on Mac OS X, so, like with GRASS, I decided to analyze the Python scripts and the sextante log to understand what happens

This time, no error in the log but no results either, no resulting shapefile in /Users/martinlaloux/sextante/tempdata. The temporary script /Users/martinlaloux/sextante/sextante_script.r has been created so the problem is again in the parameters of the Python module subprocess in Rutils.py (line 58):

command = ["R", "CMD","BATCH", "--vanilla", RUtils.getRScriptFilename(), RUtils.getConsoleOutputFilename()]
proc = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True)

The problem is shell= True with Unix systems. I have already encountered this problem many times in my work
A good explanation is given in the stackoverflow.com question: [[http://stackoverflow.com/questions/9095733/getting-output-from-python-script-in-python-tests/9096104#9096104]]

"If you're using shell=True, don't pass the program and its arguments as a list" but whith a string

- replacing the line 58 of Rutil.py by

command = "R CMD BATCH --vanilla " + RUtils.getRScriptFilename() + " "+ RUtils.getConsoleOutputFilename()
proc = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True)

works because the result of command is a string

- or
command = ["R", "CMD","BATCH", "--vanilla", RUtils.getRScriptFilename(), RUtils.getConsoleOutputFilename()]
proc = subprocess.Popen(command, stdout=subprocess.PIPE, stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True)

works also because shell=false.

The complete explanation is (from the question of stackoverflow):

"*On Unix, with shell=True: If args is a string, it specifies the command string to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself*"

I hope you understand why the original script does not work on Mac OS X, a Unix system. By modifying line 58, no more problem

History

Updated by Martin Laloux almost 2 years ago

On Unix, with shell=True: If args is a string, it specifies the command string to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself.

Updated by Victor Olaya almost 2 years ago

  • Status changed from Open to Resolved

Fixed in r301

WE already had this problem with OTB, and applied the same solution. For some reason, we kept the error in the R backend. Thanks for pointing it out and for the detailed explanation!

Updated by Giovanni Manghi over 1 year ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF