Automating inputs to linux scripts

Introduction

expect is a handy tool when you have to run a program/command which asks for input of some kind e.g. type yes/no or enter password and you want to automate it.

Installation

Run following command to install expect if it is not already installed (installation can be different for different linux distributions)

sudo apt install expect

Example

The following command will try to restart the NetworkManager, however, it needs user password to do so.

vboxuser@multix:~$ sudo systemctl restart NetworkManager
[sudo] password for vboxuser: 

We can automate the password input using expect utility

#!/bin/bash

expect <<-EOF
	set timeout -1
	spawn sudo systemctl restart NetworkManager
	expect "*password for vboxuser:"
	send "12345\r"
	expect eof
EOF

Explanation

In the above example, we are running expect command and then using here-doc to send commands to expect.

Here-document is a special purpose code block which can be used to pass multiple inputs to command. See more details here

By default, expect will wait for 10 seconds before sending the null input, set timeout -1 will set the timeout to infinite. spawn will create a new process to run the command sudo systemctl restart NetworkManager (you can also use another script file instead of command here). On the next line, expect will wait for the given pattern to match, we are matching any line ending with password for vboxuser: (note the * in the start of line). Once the pattern/line is matched, send will send the input (“12345\r” in this case). The final expect eof means expect the completion of program and exit.

Conditional Response

expect <<-EOF
	set timeout -1
	spawn sudo systemctl restart NetworkManager
	expect {
        "*password for vboxuser:" {
            send "12345\r"
        }
        "*password for otheruser:" {
            send "54321\r"
        }
    }
	expect eof
EOF

Here, a different password is sent for each user vboxuser and otheruser

Using expect with #! notation

expect can also be invoked with #! notation. The above example can be written as following in a seaprate script file. Note the #!/usr/bin/expect -f line at the start of the script

#!/usr/bin/expect -f
set timeout -1
spawn sudo systemctl restart NetworkManager
expect {
        "*password for vboxuser:" {
            send "12345\r"
        }
        "*password for otheruser:" {
            send "54321\r"
        }
}
expect eof

When using expect as script file, variables can also be set using set VariableName command and used with $VariableName. In following example passwords were extracted to separate variables (password and password2)

#!/usr/bin/expect -f
set timeout -1
set password "12345"
set password2 "54321"
spawn sudo systemctl restart NetworkManager
expect {
        "*password for vboxuser:" {
            send "$password\r"
        }
        "*password for otheruser:" {
            send "$password2\r"
        }
}
expect eof

Debugging

Debugging mode is useful to check which patterns are being matched or what reponse is being sent by expect command. It can be enabled using -d switch, see example and output below

#!/bin/bash

expect -d <<-EOF
	set timeout -1
	spawn sudo systemctl restart NetworkManager
	expect "*password for vboxuser:"
	send "12345\r"
	expect eof
EOF

Output

expect version 5.45.4
argv[0] = expect  argv[1] = -d  
set argc 0
set argv0 "expect"
set argv ""
executing commands from command file
spawn sudo systemctl restart NetworkManager
parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {8083}

expect: does "" (spawn_id exp4) match glob pattern "*password for vboxuser:"? no
[sudo] password for vboxuser: 
expect: does "[sudo] password for vboxuser: " (spawn_id exp4) match glob pattern "*password for vboxuser:"? yes
expect: set expect_out(0,string) "[sudo] password for vboxuser:"
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) "[sudo] password for vboxuser:"
send: sending "12345\r" to { exp4 }

expect: read eof
expect: set expect_out(spawn_id) "exp4"
expect: set expect_out(buffer) " \r\n"

Pro Tip

For simple cases where a command expects yes/no, yes command can be used e.g.

yes | sudo apt upgrade