Assertions

The assertions module contains classic unittest-style assert statements.

reset(report=<pedal.core.report.Report object>)

Resets (or initializes) the information about assertions.

Parameters:

report –

Collection of feedback functions related to static checks of student code.

class ensure_ast(name, at_least=1, root=None, **kwargs)

Determines if the given ast name is used anywhere, returning the node of it if it is. You can refer to Green Tree Snakes documentation for more information about AST Names:

https://greentreesnakes.readthedocs.io/en/latest/nodes.html

Parameters:

• name (str) – The name of the node.

• at_least (int) – The minimum number of times you must use this node. Defaults to 1.

condition()

Use find_all to check number of occurrences.

ensure_dataclass(name, fields=None, root=None, compliment=False, **kwargs)

Ensures that the actual definition for the given class exists. You can provide either a string and fields, or an instructor-defined version

Parameters:

• name (str or dataclass) – Either the name of the expected dataclass, or a dataclass object (not an instance).

• fields (dict) – A dictionary of field names to types.

• root (CaitNode) – The AST to search through; defaults to program root.

• compliment (str or bool) – If True, will use the compliment function to provide positive feedback. If a string, will use that string as the compliment. If False, will not provide positive feedback.

class ensure_documented_functions(root=None, **kwargs)

Checks that all of the functions in the source code are documented.

condition()

Traverse the AST to find matches.

ensure_function(name, arity=None, parameters=None, returns=None, root=None, compliment=False, **kwargs)

Checks that the function exists and has the right signature.

class ensure_function_call(name, at_least=1, root=None, **kwargs)

Determine that a function was definitely called by statically checking the students’ code.

# TODO: Allow parameter to force this to be a method xor/or function call # TODO: Allow set to be passed in for OR, or list for AND, for multiples

# TODO: also check the TIFA records to see if the function was read

Parameters:

• name (str) – The name of the function (e.g., 'sum')

• at_most (int) – The maximum number of times you are allowed to call this function. Defaults to 0.

condition()

Use find_matches to check number of occurrences.

class ensure_functions_return(exceptions=None, root=None, **kwargs)

Detects if the user has defined a function with a print statement inside.

Parameters:

exceptions – A set of function names (or a single string) of functions to allow to print (e.g., 'main').

condition(*args, **kwargs)

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

class ensure_import(name, at_least=1, root=None, **kwargs)

Verify that the given module has been imported, by statically checking the import statements in the code. The at_least parameter is ignored.

condition()

Traverse the AST to find matches.

class ensure_literal(literal, at_least=1, root=None, **kwargs)

Make sure that the given literal value does appear in the student’s code.

condition()

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

class ensure_literal_type(literal_type, at_least=1, root=None, **kwargs)

Make sure that the given type of literal value does appear in the student’s code.

condition()

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

class ensure_operation(name, at_least=1, root=None, **kwargs)

Determines if the given operator op_name is used somewhere. You can specify the operator as a string like "+" or "<<". Supports all comparison, boolean, binary, and unary operators.

Parameters:

• name (str) – The name of the function (e.g., 'sum')

• at_least (int) – The minimum number of times you must call this function. Defaults to 1.

condition()

Use find_matches to check number of occurrences.

ensure_operator

alias of ensure_operation

ensure_prints_exactly(count, **kwargs)

DEPRECATED.

Used to ensure that the student’s code calls the print function a certain number of times.

Parameters:

count (int) – The minimum and maximum number of times you can call print.

Returns:

Whether or not both the maximum and minimum was met.

Return type:

bool

class ensure_starting_code(code, root=None, **kwargs)

Detects if the given code is missing.

condition()

Traverse the AST to find matches.

files_not_handled_correctly(*filenames, muted=False)

Statically detect if files have been opened and closed correctly. This is only useful in the case of very simplistic file handling.

TODO: This could be vastly improved with line numbers and other information.

function_prints(function_name)

Determine that the print function is called within this function, by tracing its code statically. Does not actually verify that the function was ever called in practice.

Parameters:

function_name (str) – The name of the function to search.

Returns:

has_import(ast, name)

Determine if the given module name is in the code.

Parameters:

• ast (pedal.cait.cait_node.CaitNode) – The starting point to search.

• name (str) – The name of the module to match against.

Returns:

Whether or not the module name was matched.

Return type:

bool

class only_printing_variables(root=None, **kwargs)

Determines whether the user is only printing variables, as opposed to literal values.

condition()

Uses find_all to detect matches, ignoring named values.

class open_without_arguments(*args, label=None, category=None, justification=None, fields=None, field_names=None, kind=None, title=None, message=None, message_template=None, else_message=None, else_message_template=None, priority=None, valence=None, location=None, score=None, correct=None, muted=None, unscored=None, tool=None, version=None, author=None, tags=None, parent=None, report=<pedal.core.report.Report object>, delay_condition=False, activate=True, **kwargs)

Detects if the user has called the open function without any arguments.

prevent_advanced_iteration(allow_while=False, allow_for=False, allow_function=None, **kwargs)

Prevents the student from using certain advanced iteration functions and constructs. Does not currently support blocking recursion.

class prevent_ast(name, at_most=0, root=None, **kwargs)

Determines if the given ast name is not used anywhere, returning the node of it if it is. You can refer to Green Tree Snakes documentation for more information about AST Names:

https://greentreesnakes.readthedocs.io/en/latest/nodes.html

Parameters:

• name (str) – The name of the function (e.g., 'sum')

• at_most (int) – The maximum number of times you are allowed to use this AST. Defaults to 0.

condition()

Use find_all to check number of occurrences.

class prevent_embedded_answer(code, root=None, **kwargs)

Detects if the given code is present.

condition()

Traverse the AST to find matches.

class prevent_function_call(name, at_most=0, root=None, **kwargs)

Determine if a function was ever called by statically checking the students’ code.

# TODO: also check the TIFA records to see if the function was read

Parameters:

• name (str) – The name of the function (e.g., 'sum')

• at_most (int) – The maximum number of times you are allowed to call this function. Defaults to 0.

condition()

Use find_matches to check number of occurrences.

class prevent_import(name, at_most=0, root=None, **kwargs)

Verify that the given module has not been imported, by statically checking the import statements in the code. The at_most parameter is ignored.

condition()

Traverse the AST to find matches.

class prevent_literal(literal, at_most=0, root=None, **kwargs)

Make sure that the given literal value does not appear in the student’s code.

condition()

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

class prevent_literal_type(literal_type, at_most=0, root=None, **kwargs)

Make sure that the given literal value does not appear in the student’s code.

condition()

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

class prevent_operation(name, at_most=0, root=None, **kwargs)

Determines if the given operator op_name is not used anywhere. Otherwise, returns None. You can specify the operator as a string like "+" or "<<". Supports all comparison, boolean, binary, and unary operators.

Parameters:

• name (str) – The name of the function (e.g., 'sum')

• at_most (int) – The maximum number of times you are allowed to call this function. Defaults to 0.

condition()

Use find_matches to check number of occurrences.

prevent_operator

alias of prevent_operation

class prevent_printing_functions(exceptions=None, root=None, **kwargs)

Detects if the user has defined a function with a print statement inside.

Parameters:

exceptions – A set of function names (or a single string) of functions to allow to print (e.g., 'main').

condition(*args, **kwargs)

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

Runtime assertions.

TODO: assert_has_class TODO: assertGraphType, assertGraphValues TODO: assert_coverage TODO: assert_ran (latest run produced no expections)

assert_equal assert_not_equal assert_less assert_less_equal assert_greater assert_greater_equal assert_in assert_not_in assert_contains_subset assert_is assert_is_not assert_is_none assert_is_dataclass assert_is_not_dataclass assert_true assert_false assert_length_equal

assertAlmostEqual

alias of assert_equal

assertEqual

alias of assert_equal

assertFalse

alias of assert_false

assertGreater

alias of assert_greater

assertGreaterEqual

alias of assert_greater_equal

assertHasAttr

alias of assert_has_attr

assertHasFunction

alias of assert_has_function

assertHasVariable

alias of assert_has_variable

assertIn

alias of assert_in

assertIs

alias of assert_is

assertIsInstance

alias of assert_is_instance

assertIsNone

alias of assert_is_none

assertIsNot

alias of assert_is_not

assertIsNotInstance

alias of assert_not_is_instance

assertIsNotNone

alias of assert_is_not_none

assertLengthEqual

alias of assert_length_equal

assertLengthGreater

alias of assert_length_greater

assertLengthGreaterEqual

alias of assert_length_greater_equal

assertLengthLess

alias of assert_length_less

assertLengthLessEqual

alias of assert_length_less_equal

assertLengthNotEqual

alias of assert_length_not_equal

assertLess

alias of assert_less

assertLessEqual

alias of assert_less_equal

assertNotAlmostEqual

alias of assert_not_equal

assertNotEqual

alias of assert_not_equal

assertNotIn

alias of assert_not_in

assertNotOutput

alias of assert_not_output

assertNotOutputContains

alias of assert_not_output_contains

assertNotRegex

alias of assert_not_regex

assertNotType

alias of assert_not_type

assertOutput

alias of assert_output

assertOutputContains

alias of assert_output_contains

assertPrints

alias of assert_prints

assertRegex

alias of assert_regex

assertTrue

alias of assert_true

assertType

alias of assert_type

class assert_almost_equal(left, right, exact_strings=False, delta=0.001, **kwargs)

Test if the two values are almost equal; equivalent to assert_equal.

class assert_contains_subset(needles, haystack, **kwargs)

Determine if the needles are in the haystack.

condition(needles, haystack)

Tests if the needle is not in the haystack

class assert_equal(left, right, exact_strings=False, delta=0.001, **kwargs)

Determine if the left and right values are equal.

Parameters:

• exact_strings (bool) – Whether to require that strings be exactly the same, for each character. If False (the default), then strings will be normalized (lowercased, trailing decimals chopped, punctuation removed, lines are flattened, and all characters are sorted).

• delta (float) – When comparing floats, how close the values must be. If delta is None, then the default Delta will be used (.001).

condition(left, right, exact_strings, delta)

Tests if the left and right are equal

class assert_false(value, **kwargs)

Determine if the value is false.

condition(left, right)

Tests if the left evaluates to true

class assert_greater(left, right, **kwargs)

Determine if the left is greater than the right.

condition(left, right)

Tests if the left is less than or equal to the right

class assert_greater_equal(left, right, **kwargs)

Determine if the left is greater than or equal to the right.

condition(left, right)

Tests if the left is less than the right

class assert_has_attr(left, right, *args, **kwargs)

Determine if the object has the name

condition(obj, attr, exact_strings)

Tests if the regex does not match the text

class assert_has_function(sandbox, function_name, **kwargs)

Determine if the student’s code has the function.

condition(sandbox, function_name)

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

format_assertion(sandbox, function_name, contexts)

Creates a textual explanation of the difference between left and right, using any targets from the contexts. The expected_verb is the human-friendly description of the comparison being made between the two sides, while inverse_operator is the actual comparison that was performed.

Parameters:

• left (InterpolatedValue) –

• right (InterpolatedValue) –

• contexts (list[pedal.sandbox.data.SandboxContext]) –

Returns:

class assert_has_variable(sandbox, variable_name, **kwargs)

Determine if the student’s data has the name.

condition(sandbox, variable_name)

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

format_assertion(sandbox, variable_name, contexts)

Creates a textual explanation of the difference between left and right, using any targets from the contexts. The expected_verb is the human-friendly description of the comparison being made between the two sides, while inverse_operator is the actual comparison that was performed.

Parameters:

• left (InterpolatedValue) –

• right (InterpolatedValue) –

• contexts (list[pedal.sandbox.data.SandboxContext]) –

Returns:

class assert_in(needle, haystack, **kwargs)

Determine if the needle is in the haystack.

condition(needle, haystack)

Tests if the needle is not in the haystack

class assert_is(left, right, **kwargs)

Determine if the left and right values are identical.

condition(left, right)

Tests if the left and right are equal

class assert_is_dataclass(value, **kwargs)

Determine if the value is a dataclass.

condition(left, right)

Tests if the left evaluates to true

class assert_is_instance(obj, cls, **kwargs)

Determine if the obj is an instance of cls

condition(obj, cls)

Tests if the left and right are equal

class assert_is_none(value, **kwargs)

Determine if the value is None.

condition(left, right)

Tests if the left and right are equal

class assert_is_not(left, right, **kwargs)

Determine if the left and right values are not identical.

condition(left, right)

Tests if the left and right are equal

class assert_is_not_dataclass(value, **kwargs)

Determine if the value is not a dataclass.

condition(left, right)

Tests if the left evaluates to true

class assert_is_not_none(value, **kwargs)

Determine if the value is not None.

condition(left, right)

Tests if the left and right are equal

class assert_length_equal(sequence, length, **kwargs)

Determine if the sequence has the length.

condition(sequence, length)

Tests if the needle is not in the haystack

class assert_length_greater(sequence, length, **kwargs)

Determine if the sequence has greater than the length.

condition(sequence, length)

Tests if the needle is not in the haystack

class assert_length_greater_equal(sequence, length, **kwargs)

Determine if the sequence has greater than or equal to the length.

condition(sequence, length)

Tests if the needle is not in the haystack

class assert_length_less(sequence, length, **kwargs)

Determine if the sequence has less than the length.

condition(sequence, length)

Tests if the needle is not in the haystack

class assert_length_less_equal(sequence, length, **kwargs)

Determine if the sequence has less or equal than the length.

condition(sequence, length)

Tests if the needle is not in the haystack

class assert_length_not_equal(sequence, length, **kwargs)

Determine if the sequence does not have the length.

condition(sequence, length)

Tests if the needle is not in the haystack

class assert_less(left, right, **kwargs)

Determine if the left is less than the right.

condition(left, right)

Tests if the left is greater or equal

class assert_less_equal(left, right, **kwargs)

Determine if the left is less than or equal to the right.

condition(left, right)

Tests if the left is greater than the right

class assert_not_almost_equal(left, right, exact_strings=False, delta=0.001, **kwargs)

Test if the two values are not almost equal; equivalent to assert_not_equal.

class assert_not_contains_subset(needles, haystack, **kwargs)

Determine if the needles are not in the haystack.

condition(needles, haystack)

Tests if the needle is not in the haystack

class assert_not_equal(left, right, exact_strings=False, delta=0.001, **kwargs)

Determine if the left and right values are not equal.

Parameters:

• exact_strings (bool) – Whether to require that strings be exactly the same, for each character. If False (the default), then strings will be normalized (lowercased, trailing decimals chopped, punctuation removed, lines are flattened, and all characters are sorted).

• delta (float) – When comparing floats, how close the values must be. If delta is None, then the default Delta will be used (.001).

condition(left, right, exact_strings, delta)

Tests if the left and right are not equal

class assert_not_in(needle, haystack, **kwargs)

Determine if the needle is in the haystack.

condition(needle, haystack)

Tests if the needle is in the haystack

class assert_not_is_instance(obj, cls, **kwargs)

Determine if the obj is an instance of cls

condition(obj, cls)

Tests if the left and right are equal

class assert_not_output(execution, text, exact_strings=False, **kwargs)

Determine if the execution does not output text. Simply the inverse of pedal.assertions.runtime.assert_output() so check the rules there for more information.

condition(execution, text, exact_strings)

Tests if the regex does not match the text

class assert_not_output_contains(execution, text, exact_strings=False, **kwargs)

Determine if the execution outputs text

condition(execution, text, exact_strings)

Tests if the regex does not match the text

class assert_not_output_regex(pattern, execution, exact_strings=False, **kwargs)

Determine if the execution does not output text. Simply the inverse of pedal.assertions.runtime.assert_output() so check the rules there for more information.

condition(execution, text, exact_strings)

Tests if the regex does not match the text

class assert_not_regex(regex, text, **kwargs)

Determine if the regex does not match text.

condition(regex, text)

Tests if the regex does not match the text

class assert_not_type(value, expected_type, **kwargs)

condition(value, expected_type)

Tests if the left and right are equal

class assert_output(execution, text, exact_strings=False, **kwargs)

Determine if the execution outputs text. Uses the == operator to do the final comparison. In this case, you can think of the output as a single string with newlines, as opposed to a list of strings (i.e., it is retrieved with raw_output).

If the exact_strings parameter is set to be False, then output is first normalized following this strategy: * Make strings lowercase * Remove all punctuation characters * Split the string by newlines into a list * Split each individual line by spaces (aka into words) * Remove all empty lines * Sorts the lines by default order

So the default check will be fairly generous about checking output; as long as all the lines are there (in whatever order), ignoring punctuation and case, the text will be found.

condition(execution, text, exact_strings)

Tests if the regex does not match the text

class assert_output_contains(execution, text, exact_strings=False, **kwargs)

Determine if the execution outputs text anywhere. Unlike pedal.assertions.runtime.assert_output(), this function uses the in operator. If the exact_strings parameter is False, then both strings are only lowercased first (but the other normalization rules from assert_output are not applied). Can be a more flexible check since it just looks for whether the run of characters is ANYWHERE in the output. Remember that newlines are part of the output, though, so the check will not work across lines unless the text includes those newlines.

condition(execution, text, exact_strings)

Tests if the regex does not match the text

class assert_output_regex(pattern, execution, exact_strings=False, **kwargs)

Determine if the execution output matches the given regex, similar to assert_output and assert_regex.

condition(execution, text, exact_strings)

Tests if the regex does not match the text

class assert_prints(execution, text, exact_strings=False, **kwargs)

Deprecated version of assert_output

class assert_regex(regex, text, **kwargs)

Determine if the regex matches text.

condition(regex, text)

Tests if the regex matches the text

class assert_true(value, **kwargs)

Determine if the value is true.

condition(left, right)

Tests if the left evaluates to true

class assert_type(value, expected_type, **kwargs)

Same as assert_is_instance, but has a slightly different wording.

class ensure_called_uniquely(function_name, at_least=1, ignore=None, why_ignored='', **kwargs)

Verifies that the most recent executed and traced student code has at_least called the given function uniquely that number of times. In other words, it prevents students from calling the same function repeatedly WITHOUT changing the arguments.

TODO: Allow instructor to ignore certain collection of arguments TODO: Report how many calls it has seen so far.

Parameters:

• function_name (str) – The name of the function to check.

• at_least (int) – The number of calls that have to have unique arguments between them.

• ignore (set[tuple]) – A sequence of argument sets to ignore.

• why_ignored (str) – If you want to explain why you are ignoring some of the tests, you can provide some text here. For example, " because they overlap with examples you were given".

condition(unique_call_count, at_least)

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

class ensure_coverage(at_least=0.5, **kwargs)

Verifies that the most recent executed and traced student code has at_least the given ratio of covered (executed) lines.

Parameters:

at_least (float) – The ratio of covered lines. A value of 1.0 is all lines covered, 0.0 is no lines covered, and .5 is half the lines covered.

condition(coverage, at_least)

Detect if this feedback is present in the code. Defaults to true through the activate parameter.

Returns:

Whether this feedback’s condition was detected.

Return type:

bool

Unifying collection of all the commands in pedal.assertions.

class check_dataclass_error(value, **kwargs)

unit_test(function, *tests, else_message=None, else_message_template=None, score=None, partial_credit=False, assert_function=None, context=None, **kwargs)

Helper function for quickly unit testing.

• Specify exact score for each test case, individually | list[str]

• Specify exact score for each test case, with single value | str

• Specify total score for all test cases, no partial credit | False, using score

• Specify total score for all test cases, divide by # of cases passed | True, using score

Parameters:

• assert_function – Override the assert function (defaults to assert_equal)

• function (str) – The name of the function to test.

• () (**kwargs) – The test cases to run. Each test case is a tuple of the input/output pairs.

• else_message (str) – The message to present as a positive if the tests all pass.

• else_message_template (str) – The template for the else_message, to be formatted.

• score (str) – The score to give overall/per test case, depending on partial_credit.

• partial_credit (bool or str or list[str]) – Whether to give credit for each unit test as a percentage of the whole (True), or to only give points for passing all the tests (False). Defaults to False. If a list is passed, those scores will be used per test case.

• context (Sandbox or CommandBlock) – The context to run the function in.

• () –

Returns:

class wheat_chaff_game_class(name, wheats, chaffs, **kwargs)

condition()

Check that there are no errors or failures.