Sandbox

The primary goal of this module is to rerun the students code under other circumstances.

Typically, your environment will handle running the Sandbox for you. Sometimes, it will avoid running the students’ code using Pedal in favor of its own execution environment. However, they evaluate the students’ code, most environments will provide a student variable that holds information about the executed code.

class Sandbox: A Sandbox is a container where students’ code can be executed.

from pedal.sandbox import run

student = run()

student: Sandbox: The Sandbox object is a proxy for the students’ code. It is a dictionary-like object that contains all of the variables and functions that the students’ code created. It also contains a few other fields that are useful for understanding the context of the students’ code.

call(function: str, *arguments: Any, inputs: list[str] = None, target: str = '_') → Result

Calls the given function that the student has defined, using all the given arguments. You can pass in inputs to mock any builtin input calls. You can also choose the variable within the students’ code namespace that the result will be assigned to.

Note that this function returns a Sandbox Result Proxy. This is an extremely sophisticated type that attempts to perfectly imitate built-in values as much as possible. If a students’ code returns an integer, you can treat the returned value as if it were an integer (adding, subtracting, even using things like isinstance). However, extra meta information is sneakily included in the result, which allows more sophisticated output that includes the context of the call.

run(code: str, inputs: list[str] = None) → student
run(code: str, filename: str = None) → student: Runs the given arbitrary code, as you might expect from calling exec. However, handles a lot of common sandboxing issues. For example, the students’ code is in a separate thread and shouldn’t be able to modify anything about the Report object or access files on disk (unless you explicitly allow them). Code with errors will have their errors adjusted to better provide the context of what went wrong and ensure that students understand why their code is wrong and not the instructors.

evaluate(code) → Result
evaluate(code, target='_') → Result: Evaluates the given code, returning a Sandbox Result. This is an extremely sophisticated type that attempts to perfectly imitate built-in values as much as possible. If a students’ code returns an integer, you can treat the returned value as if it were an integer (adding, subtracting, even using things like isinstance). However, extra meta information is sneakily included in the result, which allows more sophisticated output that includes the context of the call.

clear_input(): Remove any existing inputs queued up for the current sandbox.

queue_input(*inputs: str)

Adds the given inputs strings to the queue of inputs that will be returned by the builtin input function. This allows you to test students’ code that involves the builtin input function, mocking whatever needs to be typed in. This does not remove any existing inputs.

queue_input('hello')
queue_input('hello', 'world')

set_input(inputs: list[str])

set_input(inputs: str)

set_input(inputs: list[str], clear=True)

Sets the given inputs strings to be the value that input will return. This allows you to test students’ code that involves the builtin input function, mocking whatever needs to be typed in.

You can provide the inputs as a single string, a list of strings, or a function that will be called to generate the inputs. Non-string values will be converted to strings using str().

If clear is True, then any existing inputs will be removed.

set_input('hello')
set_input(['hello', 'world'])

def generate_numbers(prompt: str):
    i = 0
    while True:
        yield i
        i += 1
set_input(generate_numbers)

get_input() → list[str]

Retrieves the current inputs that are available for execution.

set_input("Hello")
print(get_input())

set_input(str)
print(get_input())

clear_output(): Remove any existing output from the current sandbox.

get_output() → list[str]: Retrieves the current output that has been generated by the students’ code. The result is rstrip’d to remove any trailing whitespace, including the newlines that broke the output up in the first place.

get_raw_output() → str: Returns the raw output of the students’ code, as a string.

get_exception() → Exception | None: Returns the last exception that was raised by the students’ code, if any.

get_python_errors() → list[Feedback]: Returns any Feedback Objects that are in the runtime or syntax categories. This is useful for getting the errors that were raised by the students’ code.

clear_student_data(): Clears all of the students’ data from the current sandbox.

get_student_data() → dict[str, Any]

Retrieves the current data in the student namespace. Note that this is the data itself - modifying the dictionary will modify the data in the students’ namespace for subsequent executions!

See data for more information.

data: dict[str, Any]

All of the students’ created variables and functions will be stored in a dictionary attached to the data field. You can access it like so:

from pedal.sandbox import run
student = run('alpha = 7')
student.data['alpha']

get_sandbox() → Sandbox: Returns the current Sandbox object. This will usually be the same as the student variable.

clear_sandbox(): Clears all of the data from the current sandbox.

get_trace() → list[int]: Retrieves the list of line numbers that have been traced (recognized as executed) by all the executions so far.

get_call_arguments(function_name) → list[dict[str, any]]: Retrieves the list of arguments that have been passed to the given function name. Each argument is a dictionary mapping the parameter names to their values.

count_unique_calls(function_name) → int: Returns the number of unique calls that have been made to the given function name.

start_trace()

start_trace(tracer_style='native')

Starts tracing the execution of the students’ code. This will record the line numbers that are executed, which can be retrieved using get_trace(). By default, the tracer is already set in most environments.

The default tracer_style is 'native', which uses the Python sys.settrace function to trace the execution.

stop_trace(): Stops tracing the execution of the students’ code. This will stop recording the line numbers that are executed.

check_coverage()

Checks that all the statements in the program have been executed. This function only works when a tracer_style has been set in the sandbox, or you are using an environment that automatically traces calls (e.g., BlockPy).

If the source file was not parsed, then the first value in the tuple will be None. Otherwise, it will be a set of the line numbers that were not executed. The second value is the percentage of lines that were executed.

clear_mocks(): Clears all of the mocks that have been set up for the current sandbox.

mock_function(function_name: str, new_version)

Mocks the given function name with the given new version. The new version can be a function.

def my_sum(values):
    return 0
mock_function('sum', my_sum)

allow_function(function_name: str)

Allows the given function name to be called. This is useful for testing students’ code that uses functions that you don’t want to mock.

allow_function('sum')

block_function(function_name: str)

Blocks the given function name from being called. This is useful for testing students’ code that uses functions that you don’t want to be called.

block_function('sum')

allow_module(module_name: str)

Allows the given module name to be imported. This is useful for testing students’ code that uses modules that you don’t want to mock.

allow_module('math')

block_module(module_name: str)

Blocks the given module name from being imported. This is useful for testing students’ code that uses modules that you don’t want to be imported.

block_module('math')

mock_module(module_name: str, new_version)

mock_module(module_name: str, new_version, friendly_name=None)

Mocks the given module_name with the given new_version. The new version can be a mockedMockModule or a dictionary. The dictionary is a mapping of fields to values.

The optional friendly_name is an internal name to use to store the data for this module, accessible via Sandbox’s modules field.

def my_sum(values):
    return 0
mock_module('math', {'sum': my_sum})

get_module(module_name: str)

Loads the data for the given mocked module, if available (otherwise raises a ValueError). This is useful for getting the results of a mocked module.

get_module('plotting')

allow_real_io(): Allow the input() and print() functions to actually write to the real stdout. By default, Pedal will block this kind of writing/reading normally (although students can still use those functions).

block_real_io(): Explicitly block students from using the input() and print() functions to write/read to the real stdout. This does not prevent them from using the functions, but it does prevent the output from appearing in the real console.

class CommandBlock: A CommandBlock is a context manager that let’s you run multiple statements. These statements will be collapsed into a single context. When relevant assertions are made, the context shown will be the entire block, rather than just the last statement.