qolumbina.utils.save_tests module

qolumbina.utils.save_tests module#

The module provides utility functions to save test results of unit tests involved in ./tests/programs and parameters to JSON files.

make_test_result_collector(program_name, save_dir, scope='module')[source]#

Factory to create a pytest fixture that collects test results and writes a summary JSON with pass_rate. Only stores id and result per test case.

Parameters:

  • program_name (str) – Name of the program being tested, used in the JSON output

  • save_dir (str) – Directory where the JSON file will be saved

  • scope (Literal['session', 'package', 'module', 'class', 'function']) – Scope of the pytest fixture (default: “module”)

process_test_result(output_type, output_data, request, test_oracle_func, test_start_time, test_result_collector)[source]#

Process and save the test result using the provided oracle function.

Parameters:

  • output_type (str) – Type of the output (e.g., backend name).

  • output_data (Any) – The actual output data from the test.

  • request (FixtureRequest) – Pytest fixture request object for accessing test metadata.

  • test_oracle_func (Callable[[], None]) – A callable that performs assertions to validate the test.

  • test_start_time (float) – Timestamp when the test started.

  • test_result_collector (Tuple[Callable[[Dict[str, Any]], None], List[Dict[str, Any]]]) – A tuple containing a callable to add results and a list to store them.

Contents