If you are using Linux, then you would definitely love the shell commands.
And if you are working with Python, then you may have tried to automate things. That’s a way to save time. You may also have some bash scripts to automate things.
Python is handy to write scripts than bash. And managing Python scripts are easy compared to bash scripts. You will find it difficult to maintain the bash scripts once it’s growing.
But what if you already have bash scripts that you want to run using Python?
Is there any way to execute the bash commands and scripts in Python?
Yeah, Python has a built-in module called subprocess which is used to execute the commands and scripts inside Python scripts. Let’s see how to execute bash commands and scripts in Python scripts in detail.
Executing Bash Commands
As you may have already seen the module subprocess is used to execute the bash commands and scripts. It provides different methods and classes for the same.
There are mainly one method and one class to know about from the subprocess module. They are run and Popen. These two help us to execute the bash commands in Python scripts. Let’s see them one by one.
subprocess.run()
The method subprocess.run() will take a list of strings as a positional argument. This is mandatory as it has the bash command and arguments for it. The first item in the list is the command name and the remaining items are the arguments to the command.
Let’s see a quick example.
import subprocess
subprocess.run(["ls"])
The above script list all the items in the current working directory as the script lies. There are no arguments to the command in the above script. We have given only the bash command. We can provide additional arguments to the ls command like -l, -a, -la, etc.
Let’s see a quick example with command arguments.
import subprocess
subprocess.run(["ls", "-la"])
The above command displays all the files including hidden files along with the permissions. We have provided the argument la which displays files and directories extra information and hidden files.
We may end up making some mistakes while writing the commands. Errors will raise according to the mistakes. What if you want to capture them and use them later? Yeah, we can do that using the keyword argument stderr.
Let’s see an example.
import subprocess
result = subprocess.run(["cat", "sample.txt"], stderr=subprocess.PIPE, text=True)
print(result.stderr)
Make sure you don’t have the file with the name sample.txt in the working directory. The value to the keyword argument stderr is PIPE which helps to return the error in an object. We can access it later with the same name. And the keyword argument text helps to tell that the output should be a string.
Similarly, we can capture the output of the command using the stdout keyword argument.
import subprocess
result = subprocess.run(["echo", "Hello, World!"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(result.stdout)
subprocess.run() – input
You can give input to the commands using the input keyword argument. We will give inputs in a string format. So, we need to set the keyword argument text to True. By default, it takes it in bytes.
Let’s look at an example.
import subprocess
subprocess.run(["python3", "add.py"], text=True, input="2 3")
In the above program, the Python script add.py will take two numbers as input. We have given the input to the Python script using the input keyword argument.
subprocess.Popen()
The class subprocess.Popen() is advanced than the method subprocess.run(). It gives us more options to execute the commands. We will create an instance of the subprocess.Popen() and use it for various things like knowing the status of the command execution, getting output, giving input, etc..,
There are several methods of the class subprocess.Popen() that we need to know. Let’s see them one by one along with the code examples.
wait
It is used to wait until the completion of the execution of the command. The next lines of the Python script won’t execute until the completion of the previous command that is written after the wait method. Let’s see the example.
import subprocess
process = subprocess.Popen(["ls", "-la"])
print("Completed!")
Run the above code and observe the output. You will see that the message Completed! is printed before the execution of the command. We can avoid it using the wait method. Let’s wait till the completion of the command.
import subprocess
process = subprocess.Popen(["ls", "-la"])
process.wait()
print("Completed!")
If you see the output for the above code, then you will realize that wait is actually working. The print statement is executed after the completion of the command execution.
communicate
The method communicate is used to get the output, error and give input to the command. It returns a tuple containing output and error respectively. Let’s see an example.
import subprocess
process = subprocess.Popen(["echo", "Hello, World!"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
result = process.communicate()
print(result)subprocess.Popen() – input
We can’t pass the input to the class Popen directly. We need to use the keyword argument called stdin to give the input to the command. The instance of the class Popen will provide us stdin object. It has a method called write which is used to give the input to the command.
As we discussed earlier, it will take inputs as bytes-like objects by default. So, don’t forget to set the keyword argument text to True while creating the instance of Popen.
Let’s see an example.
import subprocess
process = subprocess.Popen(["python3", "add.py"], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
process.stdin.write("2 3")
process.stdin.close()
print(process.stdout.read())poll
The method poll is used to check whether the execution of the command is completed or not. This method will return None if the command is still executing. Let’s see an example.
import subprocess
process = subprocess.Popen(['ping', '-c 5', 'geekflare.com'], stdout=subprocess.PIPE, text=True)
while True:
output = process.stdout.readline()
if output:
print(output.strip())
result = process.poll()
if result is not None:
break
In the above code, we have used the ping command with 5 requests. There is an infinite loop that iterates until the completion of the command execution. We have used the method poll to check the status of the command execution. If the method poll returns code other than None, then the execution completes. And the infinite loop breaks.
Executing Bash Scripts
We have seen two ways to execute the commands. Now, let’s see how to execute the bash scripts in Python scripts.
The subprocess has a method called call. This method is used to execute the bash scripts. The method returns the exit code from the bash script. The default exit code for the bash scripts is 0. Let’s see an example.
Create a bash script with the name practice.sh as follows.
#!/bin/bash
echo "Hello, World!"
exit 1
Now, write a Python script execute the above bash script.
import subprocess
exit_code = subprocess.call('./practice.sh')
print(exit_code)
You will get the following output once you run the above Python script.
Hello, World!
1Conclusion
We have seen how to execute bash commands and scripts in Python. You can use them to automate things more efficiently.
Happy Coding 👨💻