Sandbox ======= The primary goal of this module is to rerun the students code under other circumstances. Typically, your environment will handle running the Sandbox for you. Sometimes, it will avoid running the students' code using Pedal in favor of its own execution environment. However, they evaluate the students' code, most environments will provide a ``student`` variable that holds information about the executed code. .. class:: Sandbox A Sandbox is a container where students' code can be executed. .. code-block:: python from pedal.sandbox import run student = run() .. data:: student :type: Sandbox The Sandbox object is a proxy for the students' code. It is a dictionary-like object that contains all of the variables and functions that the students' code created. It also contains a few other fields that are useful for understanding the context of the students' code. .. function:: call(function: str, *arguments: Any, inputs: list[str]= None, target: str="_") -> Result Calls the given ``function`` that the student has defined, using all the given ``arguments``. You can pass in ``inputs`` to mock any builtin ``input`` calls. You can also choose the variable within the students' code namespace that the result will be assigned to. Note that this function returns a Sandbox Result Proxy. This is an extremely sophisticated type that attempts to perfectly imitate built-in values as much as possible. If a students' code returns an integer, you can treat the returned value as if it were an integer (adding, subtracting, even using things like ``isinstance``). However, extra meta information is sneakily included in the result, which allows more sophisticated output that includes the context of the call. .. function:: run(code: str, inputs: list[str] = None) -> student run(code: str, filename: str = None) -> student Runs the given arbitrary code, as you might expect from calling ``exec``. However, handles a lot of common sandboxing issues. For example, the students' code is in a separate thread and shouldn't be able to modify anything about the Report object or access files on disk (unless you explicitly allow them). Code with errors will have their errors adjusted to better provide the context of what went wrong and ensure that students understand why their code is wrong and not the instructors. .. function:: evaluate(code) -> Result evaluate(code, target="_") -> Result Evaluates the given code, returning a Sandbox :py:class:`Result`. This is an extremely sophisticated type that attempts to perfectly imitate built-in values as much as possible. If a students' code returns an integer, you can treat the returned value as if it were an integer (adding, subtracting, even using things like ``isinstance``). However, extra meta information is sneakily included in the result, which allows more sophisticated output that includes the context of the call. .. function:: clear_input() Remove any existing inputs queued up for the current sandbox. .. function:: queue_input(*inputs: str) Adds the given ``inputs`` strings to the queue of inputs that will be returned by the builtin ``input`` function. This allows you to test students' code that involves the builtin ``input`` function, mocking whatever needs to be typed in. This does not remove any existing inputs. :: queue_input('hello') queue_input('hello', 'world') .. function:: set_input(inputs: list[str]) set_input(inputs: str) set_input(inputs: list[str], clear=True) Sets the given ``inputs`` strings to be the value that ``input`` will return. This allows you to test students' code that involves the builtin ``input`` function, mocking whatever needs to be typed in. You can provide the ``inputs`` as a single string, a list of strings, or a function that will be called to generate the inputs. Non-string values will be converted to strings using ``str()``. If ``clear`` is ``True``, then any existing inputs will be removed. :: set_input('hello') set_input(['hello', 'world']) def generate_numbers(prompt: str): i = 0 while True: yield i i += 1 set_input(generate_numbers) .. function:: get_input() -> list[str] Retrieves the current inputs that are available for execution. :: set_input("Hello") print(get_input()) set_input(str) print(get_input()) .. function:: clear_output() Remove any existing output from the current sandbox. .. function:: get_output() -> list[str] Retrieves the current output that has been generated by the students' code. The result is ``rstrip``'d to remove any trailing whitespace, including the newlines that broke the output up in the first place. .. function:: get_raw_output() -> str Returns the raw output of the students' code, as a string. .. function:: get_exception() -> Exception | None Returns the last exception that was raised by the students' code, if any. .. function:: get_python_errors() -> list[Feedback] Returns any Feedback Objects that are in the ``runtime`` or ``syntax`` categories. This is useful for getting the errors that were raised by the students' code. .. function:: clear_student_data() Clears all of the students' data from the current sandbox. .. function:: get_student_data() -> dict[str, Any] Retrieves the current data in the student namespace. Note that this is the data itself - modifying the dictionary will modify the data in the students' namespace for subsequent executions! See :py:data:`~data` for more information. .. data:: data: dict[str, Any] All of the students' created variables and functions will be stored in a dictionary attached to the ``data`` field. You can access it like so: .. code-block:: python from pedal.sandbox import run student = run('alpha = 7') student.data['alpha'] .. function:: get_sandbox() -> Sandbox Returns the current Sandbox object. This will usually be the same as the ``student`` variable. .. function:: clear_sandbox() Clears all of the data from the current sandbox. .. function:: get_trace() -> list[int] Retrieves the list of line numbers that have been traced (recognized as executed) by all the executions so far. .. function:: get_call_arguments(function_name) -> list[dict[str, any]] Retrieves the list of arguments that have been passed to the given function name. Each argument is a dictionary mapping the parameter names to their values. .. function:: count_unique_calls(function_name) -> int Returns the number of unique calls that have been made to the given function name. .. function:: start_trace() start_trace(tracer_style='native') Starts tracing the execution of the students' code. This will record the line numbers that are executed, which can be retrieved using :py:func:`get_trace`. By default, the tracer is already set in most environments. The default ``tracer_style`` is ``'native'``, which uses the Python ``sys.settrace`` function to trace the execution. .. function:: stop_trace() Stops tracing the execution of the students' code. This will stop recording the line numbers that are executed. .. function:: check_coverage() -> tuple(set[int], float) Checks that all the statements in the program have been executed. This function only works when a tracer_style has been set in the sandbox, or you are using an environment that automatically traces calls (e.g., BlockPy). If the source file was not parsed, then the first value in the tuple will be None. Otherwise, it will be a set of the line numbers that were not executed. The second value is the percentage of lines that were executed. .. function:: clear_mocks() Clears all of the mocks that have been set up for the current sandbox. .. function:: mock_function(function_name: str, new_version) Mocks the given function name with the given new version. The new version can be a function. :: def my_sum(values): return 0 mock_function('sum', my_sum) .. function:: allow_function(function_name: str) Allows the given function name to be called. This is useful for testing students' code that uses functions that you don't want to mock. :: allow_function('sum') .. function:: block_function(function_name: str) Blocks the given function name from being called. This is useful for testing students' code that uses functions that you don't want to be called. :: block_function('sum') .. function:: allow_module(module_name: str) Allows the given module name to be imported. This is useful for testing students' code that uses modules that you don't want to mock. :: allow_module('math') .. function:: block_module(module_name: str) Blocks the given module name from being imported. This is useful for testing students' code that uses modules that you don't want to be imported. :: block_module('math') .. function:: mock_module(module_name: str, new_version) mock_module(module_name: str, new_version, friendly_name=None) Mocks the given ``module_name`` with the given ``new_version``. The new version can be a :py:class:`~pedal.sandbox.mockedMockModule` or a dictionary. The dictionary is a mapping of fields to values. The optional ``friendly_name`` is an internal name to use to store the data for this module, accessible via Sandbox's ``modules`` field. :: def my_sum(values): return 0 mock_module('math', {'sum': my_sum}) .. function:: get_module(module_name: str) Loads the data for the given mocked module, if available (otherwise raises a ``ValueError``). This is useful for getting the results of a mocked module. :: get_module('plotting') .. function:: allow_real_io() Allow the input() and print() functions to actually write to the real stdout. By default, Pedal will block this kind of writing/reading normally (although students can still use those functions). .. function:: block_real_io() Explicitly block students from using the input() and print() functions to write/read to the real stdout. This does not prevent them from using the functions, but it does prevent the output from appearing in the real console. .. class:: CommandBlock A CommandBlock is a context manager that let's you run multiple statements. These statements will be collapsed into a single context. When relevant assertions are made, the context shown will be the entire block, rather than just the last statement.