What to expect in pexpect? – Advanced examples

I remember how frustrated I was at the beginning of my journey with pexpect. I never knew what string to put as an expect argument, and even if I knew, the pexpect wasn’t catching it. Today we will go through advanced pexpect examples covering a method that helps in such a situation.

Workflow overview

In today’s script, we will try to gather a show clock output from the Cisco IOS router, but in various circumstances, which will lead to different script behaviors.

Basic scenario

Let’s take a look at the easiest first, we’re creating a pexpect session, and executing a show clock command right after logging.

import pexpect
import time

username = "admin"
password = "admin"
ip = "10.0.0.2"
prompt = "Router#"
clear_buffer = ".+"

child = pexpect.spawn(f"ssh {username}@{ip}")
child.expect("Password:")
child.sendline(password)
child.expect(prompt)
child.sendline("show clock")
child.expect(prompt)
print(f"Printing output for the first time:\n{child.before.decode()}")

Let’s take a look at the output

Printing output for the first time:
show clock
.09:30:08.775 UTC Tue Jun 13 2023

Everything is going smoothly, so let’s complicate it a little bit.

Unexpected console state

What if we would have a console that is in an unexpected state? Let’s simulate it by sending three new lines to the console before executing show clock command.

child.sendline("\r")
child.sendline("\r")
child.sendline("\r")
child.sendline("show clock")
time.sleep(2)
child.expect(prompt)
print(f"Printing output for the second time:\n{child.before.decode()}")
print(f"Reamining content of the buffer:\n{child.buffer.decode()}")

Now, let’s take a look at the result.

Printing output for the second time:


Reamining content of the buffer:

Router#
Router#
Router#
Router#
Router#show clock
.09:30:08.986 UTC Tue Jun 13 2023
Router#

It’s starting to get messy here. Till this time, we had our desired output in the child.before object, but it’s not there.

If we take a look at the pexpect object buffer (child.buffer), we can see, that our output is there, but also with additional router prompts.

Why we’re not seeing output and why it’s present in the buffer object? Let’s dive straight into pexpect core principles.

expect() mechanics

During pexpect session, it gathers all the output to the buffer. This means, that if you send a command to the device with sendline method, all the output will be present at first in the buffer object. Then you can check if something is present in the buffer by issuing expect method. If it is, everything that’s before this matched string is moved from the child.buffer to the child.before object. What’s worth mentioning is when you have multiple occurrences of the expected string in the child.buffer, only the first occurrence is matched and moved to the child.before. You can think of it as a cake. child.buffer is a sweet strawberry cake, and by issuing a expect method, you can cut a piece of it, and move it to the other plate(child.before).

Let’s move this theory to the real example.

child = pexpect.spawn(f"ssh {username}@{ip}")
child.expect("Password:")
child.sendline(password)
child.expect(prompt)
child.sendline("show clock")
child.expect(prompt)

In the first code snipped, we’re constantly clearing a buffer after sending a command. That’s why we’re catching the right pieces of output at a time.

The situation is slightly different in the second example.

child.sendline("\r")
child.sendline("\r")
child.sendline("\r")
child.sendline("show clock")
time.sleep(2)
child.expect(prompt)
print(f"Printing output for the second time:\n{child.before.decode()}")
print(f"Reamining content of the buffer:\n{child.buffer.decode()}")

In this case, we have three new lines, and then we’re sending a show clock command. In the buffer, we will have multiple router prompts (because we sent newlines), and then the output of show clock, which ends with another prompt.

When we’re executing first expect(prompt), we’re getting literally none in the output.

Printing output for the second time:

It’s because we expected a prompt, so in the buffer, the first Router# was matched. There was nothing before it, that’s why the child.before object is empty. Still, in the buffer, we have leftovers from the sendlines, when we print it, that’s what we’re getting.

Reamining content of the buffer:

Router#
Router#
Router#
Router#
Router#show clock
.09:30:08.986 UTC Tue Jun 13 2023
Router#

As we can see, all the outputs are still there.

As we can observe, it’s better to have a clean buffer before sending a command from which we want to grab output, because our expect can match not the portion of the buffer that we want.

Clearing buffer

Clearing a buffer isn’t a big deal. All we have to do is to send a expect, that will match everything. To do that, we have to reach for regex expressions. I’m using “.+”, dot stands for any character, and + states that there are one or more of those.

Let’s append some code that clears the buffer, sends again show clock, and tries to grab an output.

child.expect(clear_buffer)
print(f"Buffer content after cleaning:\n{child.buffer.decode()}")
child.sendline("show clock")
child.expect(prompt)
print(f"Printing output after clearing the buffer:\n{child.before.decode()}")
child.close()

The output is as follows.

Buffer content after cleaning:

Printing output after clearing the buffer:
show clock
.09:30:11.046 UTC Tue Jun 13 2023

The buffer is empty, and we got the command output.

This way you can use the cleaning buffer before sending and retrieving outputs from commands if you’re not sure what’s the state of the buffer.

Sometimes, however, we need to take a look at what’s in the buffer. Especially during script development. That’s what we will do in the next section.

Inspecting buffer

It may seem to be hard because to refresh the buffer object, we need to use a expect method.

But what to expect in such a situation?

If we don’t know, we will most likely receive a Timeout exception, so we won’t be able to read what’s there.

The Timeout exception is a key here. If we pass a Timeout exception to a expect method, Python won’t raise it! Moreover, the child.buffer variable won’t be changed, because the expect method won’t match anything. It’s a simple trick to look at what’s there.

Let’s go through an example.

child = pexpect.spawn(f"ssh {username}@{ip}")
child.expect("Password:")
child.sendline(password)
child.expect(prompt)
child.sendline("\r")
child.sendline("\r")
child.sendline("\r")
child.sendline("show clock")
time.sleep(2)
child.expect(pexpect.TIMEOUT, timeout=0)
print(f"Buffer content after refresh:\n{child.buffer.decode()}")

Here we have a similar situation to the previous ones. A couple of new lines were sent, followed by show clock command. In this situation, if we want to take a look at what’s in the buffer, we’re calling expect with pexpect.TIMEOUT as an argument. It’s also worth overwriting the default timeout value (which is 30 seconds), not to wait such a long time. Passing 0 will give us an instant execution. Then we can take a look at what’s in the child.buffer.

Buffer content after refresh:

Router#
Router#
Router#
Router#
Router#
Router#show clock
.09:30:15.867 UTC Tue Jun 13 2023
Router#

As you can see, we have a full buffer content.

Keep in mind, that output that you’re seeing in the console is formatted by the print function. If you want to see a raw output, you can remove decode().

b'\r\nRouter#\r\nRouter#\r\nRouter#\r\nRouter#\r\nRouter#\r\nRouter#show clock\r\n09:30:15.867 UTC Tue Jun 13 2023\r\nRouter#'

Based on that, you have more visibility of what’s going on underneath.

Here’s the complete code.

import pexpect
import time

username = "admin"
password = "admin"
ip = "10.0.1.2"
prompt = "Router#"
clear_buffer = ".+"

child = pexpect.spawn(f"ssh {username}@{ip}")
child.expect("Password:")
child.sendline(password)
child.expect(prompt)
child.sendline("show clock")
child.expect(prompt)
print(f"Printing output for the first time:\n{child.before.decode()}")

child.sendline("\r")
child.sendline("\r")
child.sendline("\r")
child.sendline("show clock")
time.sleep(2)
child.expect(prompt)
print(f"Printing output for the second time:\n{child.before.decode()}")
print(f"Reamining content of the buffer:\n{child.buffer.decode()}")

child.expect(clear_buffer)
print(f"Buffer content after cleaning:\n{child.buffer.decode()}")
child.sendline("show clock")
child.expect(prompt)
print(f"Printing output after clearing the buffer:\n{child.before.decode()}")
child.close()

child = pexpect.spawn(f"ssh {username}@{ip}")
child.expect("Password:")
child.sendline(password)
child.expect(prompt)
child.sendline("\r")
child.sendline("\r")
child.sendline("\r")
child.sendline("show clock")
time.sleep(2)
child.expect(pexpect.TIMEOUT, timeout=0)
print(f"Buffer content after refresh:\n{child.buffer.decode()}")

You can also access it via Github