# NAME

App::Test::Generator - Fuzz Testing, Mutation Testing, LCSAJ Metrics and Test Dashboard for Perl modules

# VERSION

Version 0.35

# SYNOPSIS

`App::Test::Generator` is a suite to help the testing of CPAN modules.
It consists of 4 modules:

- Fuzz Tester
- Mutation Testing
- LCSAJ Metrics
- Test Dashboard

From the command line:

    # Takes the formal definition of a routine, creates tests against that routine, and runs the test
    fuzz-harness-generator -r t/conf/add.yml

    # Attempt to create a formal definition from a routine package, then run tests against that formal definition
    # This is the holy grail of automatic test generation, just by looking at the source code
    extract-schemas bin/extract-schemas lib/Sample/Module.pm && fuzz-harness-generator -r schemas/greet.yaml

From Perl:

    use App::Test::Generator qw(generate);

    # Generate to STDOUT
    App::Test::Generator->generate("t/conf/add.yml");

    # Generate directly to a file
    App::Test::Generator->generate('t/conf/add.yml', 't/add_fuzz.t');

    # Holy grail mode - read a Perl file, generate tests, and run them
    # This is a long way away yet, but see t/schema_input.t for a proof of concept
    my $extractor = App::Test::Generator::SchemaExtractor->new(
      input_file => 'Foo.pm',
      output_dir => $dir
    );
    my $schemas = $extractor->extract_all();
    foreach my $schema(keys %{$schemas}) {
      my $tempfile = '/var/tmp/foo.t';    # Use File::Temp in real life
      App::Test::Generator->generate(
        schema => $schemas->{$schema},
        output_file => $tempfile,
      );
      system("$^X -I$dir $tempfile");
      unlink $tempfile;
    }

# OVERVIEW

This module takes a formal input/output specification for a routine or
method and automatically generates test cases. In effect, it allows you
to easily add comprehensive black-box tests in addition to the more
common white-box tests that are typically written for CPAN modules and other
subroutines.

The generated tests combine:

- Random fuzzing based on input types
- Deterministic edge cases for min/max constraints
- Static corpus tests defined in Perl or YAML

This approach strengthens your test suite by probing both expected and
unexpected inputs, helping you to catch boundary errors, invalid data
handling, and regressions without manually writing every case.

# DESCRIPTION

This module implements the logic behind [fuzz-harness-generator](https://metacpan.org/pod/fuzz-harness-generator).
It parses configuration files (fuzz and/or corpus YAML), and
produces a ready-to-run `.t` test script to run through `prove`.

It reads configuration files in any format,
and optional YAML corpus files.
All of the examples in this documentation are in `YAML` format,
other formats may not work as they aren't so heavily tested.
It then generates a [Test::Most](https://metacpan.org/pod/Test%3A%3AMost)-based fuzzing harness combining:

- Randomized fuzzing of inputs (with edge cases)
- Optional static corpus tests from Perl `%cases` or YAML file (`yaml_cases` key)
- Functional or OO mode (via `$new`)
- Reproducible runs via `$seed` and configurable iterations via `$iterations`

# MUTATION-GUIDED TEST GENERATION

`App::Test::Generator` includes a pipeline that automatically closes the
feedback loop between mutation testing, schema extraction, and fuzz
testing. The goal is that surviving mutants drive the creation of new
tests that kill them on the next run, without manual intervention.

## The Pipeline

    mutation survivor
        |
        v
    SchemaExtractor extracts the schema for the enclosing sub
        |
        v
    Schema augmented with boundary values from the mutant
        |
        v
    Augmented schema written to t/conf/
        |
        v
    t/fuzz.t picks up the new schema and runs fuzz tests
        |
        v
    Mutation killed on next run

## How to Use It

The pipeline is driven by three flags passed to
`bin/test-generator-index`, which is invoked automatically by
`bin/generate-test-dashboard` on each CI push.

### Step 1: Generate TODO stubs for all survivors

    bin/test-generator-index --generate_mutant_tests=t

Produces `t/mutant_YYYYMMDD_HHMMSS.t` containing:

- TODO stubs for HIGH and MEDIUM difficulty survivors, with
boundary value suggestions, environment variable hints, and the
enclosing subroutine name for navigation context.
- Comment-only hints for LOW difficulty survivors.

Multiple mutations on the same source line are deduplicated into one
stub. One good test kills all variants on that line.

### Step 2: Generate runnable schemas for NUM\_BOUNDARY survivors

    bin/test-generator-index \
        --generate_mutant_tests=t \
        --generate_test=mutant

For each NUM\_BOUNDARY survivor, calls
[App::Test::Generator::SchemaExtractor](https://metacpan.org/pod/App%3A%3ATest%3A%3AGenerator%3A%3ASchemaExtractor) to extract the schema for
the enclosing subroutine. If the confidence level is sufficient, the
schema is augmented with the boundary value from the mutant (plus one
value either side) and written to `t/conf/` as a runnable YAML file.
["fuzz.t" in t](https://metacpan.org/pod/t#fuzz.t) picks it up automatically on the next test run.

Falls back to a TODO stub if:

- SchemaExtractor cannot parse the file
- The enclosing sub cannot be determined
- The extracted schema confidence is `very_low` or `none`

### Step 3: Augment existing schemas with survivor boundary values

    bin/test-generator-index \
        --generate_mutant_tests=t \
        --generate_test=mutant \
        --generate_fuzz

Scans `t/conf/` for existing YAML schema files (hand-written or
previously generated) and writes augmented copies with boundary values
from surviving NUM\_BOUNDARY mutants merged in. The original schema is
never modified. Augmented copies are written as
`t/conf/mutant_fuzz_YYYYMMDD_HHMMSS_FUNCTION.yml` and picked up
automatically by `t/fuzz.t`.

Schemas whose filename already starts with `mutant_fuzz_` are skipped
to prevent cascading augmentation. Schemas with no matching survivors
are skipped, with a note if `--verbose` is active.

### Putting It All Together

The recommended invocation in `bin/generate-test-dashboard`
Step 7 runs all three stages together:

    bin/test-generator-index \
        --generate_mutant_tests=t \
        --generate_test=mutant \
        --generate_fuzz

The GitHub Actions workflow in `.github/workflows/dashboard.yml`
then commits any new `t/mutant_*.t` and `t/conf/mutant_*.yml` files
to the repository so they accumulate over time as the test suite
improves.

## Confidence Levels

[App::Test::Generator::SchemaExtractor](https://metacpan.org/pod/App%3A%3ATest%3A%3AGenerator%3A%3ASchemaExtractor) assigns a confidence level
to each extracted schema:

- `high` / `medium` / `low` - Schema is used for test generation
- `very_low` / `none` - Falls back to TODO stub

Confidence is based on how much type and constraint information could
be inferred from the source code and its POD documentation. Methods
with explicit parameter validation ([Params::Validate::Strict](https://metacpan.org/pod/Params%3A%3AValidate%3A%3AStrict),
[Params::Get](https://metacpan.org/pod/Params%3A%3AGet)) or comprehensive POD will produce higher-confidence
schemas.

## Files Produced

- `t/mutant_YYYYMMDD_HHMMSS.t`

    TODO stub file for all survivors. Committed to the repository by the
    GitHub Actions workflow.

- `t/conf/mutant_MODNAME_FUNCTION_YYYYMMDD_HHMMSS.yml`

    Runnable YAML schema for a NUM\_BOUNDARY survivor where SchemaExtractor
    confidence was sufficient. Picked up by `t/fuzz.t`.

- `t/conf/mutant_fuzz_YYYYMMDD_HHMMSS_FUNCTION.yml`

    Augmented copy of an existing schema with survivor boundary values
    merged in. Picked up by `t/fuzz.t`.

## See Also

- [App::Test::Generator::SchemaExtractor](https://metacpan.org/pod/App%3A%3ATest%3A%3AGenerator%3A%3ASchemaExtractor) - Schema extraction
from Perl source code
- ["test-generator-index" in bin](https://metacpan.org/pod/bin#test-generator-index) - Dashboard generator and
pipeline driver
- ["generate-test-dashboard" in bin](https://metacpan.org/pod/bin#generate-test-dashboard) - Full pipeline runner

# CONFIGURATION

The configuration file,
for each set of tests to be produced,
is a file containing a schema that can be read by [Config::Abstraction](https://metacpan.org/pod/Config%3A%3AAbstraction).

## SCHEMA

The schema is split into several sections.

### `%input` - input params with keys => type/optional specs

When using named parameters

    input:
      name:
        type: string
        optional: false
      age:
        type: integer
        optional: true

Supported basic types used by the fuzzer: `string`, `integer`, `float`, `number`, `boolean`, `arrayref`, `hashref`.
See also [Params::Validate::Strict](https://metacpan.org/pod/Params%3A%3AValidate%3A%3AStrict).
You can add more custom types using properties.

For routines with one unnamed parameter

    input:
      type: string

For routines with more than one named parameter, use the `position` keyword.

    module: Math::Simple::MinMax
    fuction: max

    input:
      left:
        type: number
        position: 0
      right:
        type: number
        position: 1

    output:
      type: number

The keyword `undef` is used to indicate that the `function` takes no arguments.

### `%output` - output param types for [Return::Set](https://metacpan.org/pod/Return%3A%3ASet) checking

    output:
      type: string

If the output hash contains the key \_STATUS, and if that key is set to DIES,
the routine should die with the given arguments; otherwise, it should live.
If it's set to WARNS,
the routine should warn with the given arguments.
The output can be set to the string 'undef' if the routine should return the undefined value:

    ---
    module: Scalar::Util
    function: blessed

    input:
      type: string

    output: undef

The keyword `undef` is used to indicate that the `function` returns nothing.

### `%config` - optional hash of configuration.

The current supported variables are

- `close_stdin`

    Tests should not attempt to read from STDIN (default: 1).
    This is ignored on Windows, when never closes STDIN.

- `test_nuls`, inject NUL bytes into strings (default: 1)

    With this test enabled, the function is expected to die when a NUL byte is passed in.

- `test_undef`, test with undefined value (default: 1)
- `test_empty`, test with empty strings (default: 1)
- `test_non_ascii`, test with strings that contain non ascii characters (default: 1)
- `timeout`, ensure tests don't hang (default: 10)

    Setting this to 0 disables timeout testing.

- `dedup`, fuzzing can create duplicate tests, go some way to remove duplicates (default: 1)
- `properties`, enable [Test::LectroTest](https://metacpan.org/pod/Test%3A%3ALectroTest) Property tests (default: 0)

    \*item \* `test_security`, send some security string based tests (default: 0)

All values default to `true`.

### `%accessor` - this is an accessor routine

    accessor:
      property: ua
      type: getset

Has two mandatory elements:

- `property`

    The name of the property in the object that the routine controls.

- `type`

    One of `getter`, `setter`, `getset`.

### `%transforms` - list of transformations from input sets to output sets

Transforms allow you to define how input data should be transformed into output data.
This is useful for testing functions that convert between formats, normalize data,
or apply business logic transformations on a set of data to different set of data.
It takes a list of subsets of the input and output definitions,
and verifies that data from each input subset is correctly transformed into data from the matching output subset.

#### Transform Validation Rules

For each transform:

- 1. Generate test cases using the transform's input schema
- 2. Call the function with those inputs
- 3. Validate the output matches the transform's output schema
- 4. If output has a specific 'value', check exact match
- 5. If output has constraints (min/max), validate within bounds

#### Example 1

    ---
    module: builtin
    function: abs

    config:
      test_undef: no
      test_empty: no
      test_nuls: no
      test_non_ascii: no

    input:
      number:
        type: number
        position: 0

    output:
      type: number
      min: 0

    transforms:
      positive:
        input:
          number:
            type: number
            position: 0
            min: 0
        output:
          type: number
          min: 0
      negative:
        input:
          number:
            type: number
            position: 0
            max: 0
        output:
          type: number
          min: 0
      error:
        input:
          undef
        output:
          _STATUS: DIES

If the output hash contains the key \_STATUS, and if that key is set to DIES,
the routine should die with the given arguments; otherwise, it should live.
If it's set to WARNS, the routine should warn with the given arguments.

The keyword `undef` is used to indicate that the `function` returns nothing.

#### Example 2

    ---
    module: Math::Utils
    function: normalize_number

    input:
      value:
        type: number
        position: 0

    output:
      type: number

    transforms:
      positive_stays_positive:
        input:
          value:
            type: number
            min: 0
            max: 1000
        output:
          type: number
          min: 0
          max: 1

      negative_becomes_zero:
        input:
          value:
            type: number
            max: 0
        output:
          type: number
          value: 0

      preserves_zero:
        input:
          value:
            type: number
            value: 0
        output:
          type: number
          value: 0

### `$module`

The name of the module (optional).

Using the reserved word `builtin` means you're testing a Perl builtin function.

If omitted, the generator will guess from the config filename:
`My-Widget.conf` -> `My::Widget`.

### `$function`

The function/method to test.

This defaults to `run`.

### `%new`

An optional hashref of args to pass to the module's constructor.

    new:
      api_key: ABC123
      verbose: true

To ensure `new()` is called with no arguments, you still need to define new, thus:

    module: MyModule
    function: my_function

    new:

### `%cases`

An optional Perl static corpus, when the output is a simple string (expected => \[ args... \]).

Maps the expected output string to the input and \_STATUS

    cases:
      ok:
        input: ping
        _STATUS: OK
      error:
        input: ""
        _STATUS: DIES

### `$yaml_cases` - optional path to a YAML file with the same shape as `%cases`.

### `$seed`

An optional integer.
When provided, the generated `t/fuzz.t` will call `srand($seed)` so fuzz runs are reproducible.

### `$iterations`

An optional integer controlling how many fuzz iterations to perform (default 30).

### `%edge_cases`

An optional hash mapping of extra values to inject.

        # Two named parameters
        edge_cases:
                name: [ '', 'a' x 1024, \"\x{263A}" ]
                age: [ -1, 0, 99999999 ]

        # Takes a string input
        edge_cases: [ 'foo', 'bar' ]

Values can be strings or numbers; strings will be properly quoted.
Note that this only works with routines that take named parameters.

### `%type_edge_cases`

An optional hash mapping types to arrayrefs of extra values to try for any field of that type:

        type_edge_cases:
                string: [ '', ' ', "\t", "\n", "\0", 'long' x 1024, chr(0x1F600) ]
                number: [ 0, 1.0, -1.0, 1e308, -1e308, 1e-308, -1e-308, 'NaN', 'Infinity' ]
                integer: [ 0, 1, -1, 2**31-1, -(2**31), 2**63-1, -(2**63) ]

### `%edge_case_array`

Specify edge case values for routines that accept a single unnamed parameter.
This is specifically designed for simple functions that take one argument without a parameter name.
These edge cases supplement the normal random string generation, ensuring specific problematic values are always tested.
During fuzzing iterations, there's a 40% probability that a test case will use a value from edge\_case\_array instead of randomly generated data.

    ---
    module: Text::Processor
    function: sanitize

    input:
      type: string
      min: 1
      max: 1000

    edge_case_array:
      - "<script>alert('xss')</script>"
      - "'; DROP TABLE users; --"
      - "\0null\0byte"
      - "emoji😊test"
      - ""
      - " "

    seed: 42
    iterations: 30

### Semantic Data Generators

For property-based testing with [Test::LectroTest](https://metacpan.org/pod/Test%3A%3ALectroTest),
you can use semantic generators to create realistic test data.

`unix_timestamp` is currently fully supported,
other fuzz testing support for `semantic` entries is being developed.

    input:
      email:
        type: string
        semantic: email

      user_id:
        type: string
        semantic: uuid

      phone:
        type: string
        semantic: phone_us

#### Available Semantic Types

- `email` - Valid email addresses (user@domain.tld)
- `url` - HTTP/HTTPS URLs
- `uuid` - UUIDv4 identifiers
- `phone_us` - US phone numbers (XXX-XXX-XXXX)
- `phone_e164` - International E.164 format (+XXXXXXXXXXXX)
- `ipv4` - IPv4 addresses (0.0.0.0 - 255.255.255.255)
- `ipv6` - IPv6 addresses
- `username` - Alphanumeric usernames with \_ and -
- `slug` - URL slugs (lowercase-with-hyphens)
- `hex_color` - Hex color codes (#RRGGBB)
- `iso_date` - ISO 8601 dates (YYYY-MM-DD)
- `iso_datetime` - ISO 8601 datetimes (YYYY-MM-DDTHH:MM:SSZ)
- `semver` - Semantic version strings (major.minor.patch)
- `jwt` - JWT-like tokens (base64url format)
- `json` - Simple JSON objects
- `base64` - Base64-encoded strings
- `md5` - MD5 hashes (32 hex chars)
- `sha256` - SHA-256 hashes (64 hex chars)
- `unix_timestamp`

## EDGE CASE GENERATION

In addition to purely random fuzz cases, the harness generates
deterministic edge cases for parameters that declare `min`, `max` or `len` in their schema definitions.

For each constraint, three edge cases are added:

- Just inside the allowable range

    This case should succeed, since it lies strictly within the bounds.

- Exactly on the boundary

    This case should succeed, since it meets the constraint exactly.

- Just outside the boundary

    This case is annotated with `_STATUS = 'DIES'` in the corpus and
    should cause the harness to fail validation or croak.

Supported constraint types:

- `number`, `integer`, `float`

    Uses numeric values one below, equal to, and one above the boundary.

- `string`

    Uses strings of lengths one below, equal to, and one above the boundary.

- `arrayref`

    Uses references to arrays of with the number of elements one below, equal to, and one above the boundary.

- `hashref`

    Uses hashes with key counts one below, equal to, and one above the
    boundary (`min` = minimum number of keys, `max` = maximum number
    of keys).

- `memberof` - arrayref of allowed values for a parameter

    This example is for a routine called `input()` that takes two arguments: `status` and `level`.
    `status` is a string that must have the value `ok`, `error` or `pending`.
    The `level` argument is an integer that must be one of `1`, `5` or `111`.

        ---
        input:
          status:
            type: string
            memberof:
              - ok
              - error
              - pending
          level:
            type: integer
            memberof:
              - 1
              - 5
              - 111

    The generator will automatically create test cases for each allowed value (inside the member list),
    and at least one value outside the list (which should die or `croak`, `_STATUS = 'DIES'`).
    This works for strings, integers, and numbers.

- `enum` - synonym of `memberof`
- `boolean` - automatic boundary tests for boolean fields

        input:
          flag:
            type: boolean

    The generator will automatically create test cases for 0 and 1; true and false; off and on, and values that should trigger `_STATUS = 'DIES'`.

These edge cases are inserted automatically, in addition to the random
fuzzing inputs, so each run will reliably probe boundary conditions
without relying solely on randomness.

# EXAMPLES

See the files in `t/conf` for examples.

## Adding Scheduled fuzz Testing with GitHub Actions to Your Code

To automatically create and run tests on a regular basis on GitHub Actions,
you need to create a configuration file for each method and subroutine that you're testing,
and a GitHub Actions configuration file.

This example takes you through testing the online\_render method of [HTML::Genealogy::Map](https://metacpan.org/pod/HTML%3A%3AGenealogy%3A%3AMap).

### t/conf/online\_render.yml

    ---

    module: HTML::Genealogy::Map
    function: onload_render

    input:
      gedcom:
        type: object
        can: individuals
      geocoder:
        type: object
        can: geocode
      debug:
        type: boolean
        optional: true
      google_key:
        type: string
        optional: true
        min: 39
        max: 39
        matches: "^AIza[0-9A-Za-z_-]{35}$"

    config:
      test_undef: 0

### .github/actions/fuzz.t

    ---
    name: Fuzz Testing

    permissions:
      contents: read

    on:
      push:
        branches: [main, master]
      pull_request:
        branches: [main, master]
      schedule:
        - cron: '29 5 14 * *'

    jobs:
      generate-fuzz-tests:
        strategy:
          fail-fast: false
          matrix:
            os:
              - macos-latest
              - ubuntu-latest
              - windows-latest
            perl: ['5.42', '5.40', '5.38', '5.36', '5.34', '5.32', '5.30', '5.28', '5.22']

        runs-on: ${{ matrix.os }}
        name: Fuzz testing with perl ${{ matrix.perl }} on ${{ matrix.os }}

        steps:
          - uses: actions/checkout@v5

          - name: Set up Perl
            uses: shogo82148/actions-setup-perl@v1
            with:
              perl-version: ${{ matrix.perl }}

          - name: Install App::Test::Generator this module's dependencies
            run: |
              cpanm App::Test::Generator
              cpanm --installdeps .

          - name: Make Module
            run: |
              perl Makefile.PL
              make
            env:
              AUTOMATED_TESTING: 1
              NONINTERACTIVE_TESTING: 1

          - name: Generate fuzz tests
            run: |
              mkdir t/fuzz
              find t/conf -name '*.yml' | while read config; do
                test_name=$(basename "$config" .conf)
                fuzz-harness-generator "$config" > "t/fuzz/${test_name}_fuzz.t"
              done

          - name: Run generated fuzz tests
            run: |
              prove -lr t/fuzz/
            env:
              AUTOMATED_TESTING: 1
              NONINTERACTIVE_TESTING: 1

## Fuzz Testing your CPAN Module

Running fuzz tests when you run `make test` in your CPAN module.

Create a directory &lt;t/conf> which contains the schemas.

Then create this file as &lt;t/fuzz.t>:

    #!/usr/bin/env perl

    use strict;
    use warnings;

    use FindBin qw($Bin);
    use IPC::Run3;
    use IPC::System::Simple qw(system);
    use Test::Needs 'App::Test::Generator';
    use Test::Most;

    my $dirname = "$Bin/conf";

    if((-d $dirname) && opendir(my $dh, $dirname)) {
          while (my $filename = readdir($dh)) {
                  # Skip '.' and '..' entries and vi temporary files
                  next if ($filename eq '.' || $filename eq '..') || ($filename =~ /\.swp$/);

                  my $filepath = "$dirname/$filename";

                  if(-f $filepath) {      # Check if it's a regular file
                          my ($stdout, $stderr);
                          run3 ['fuzz-harness-generator', '-r', $filepath], undef, \$stdout, \$stderr;

                          ok($? == 0, 'Generated test script exits successfully');

                          if($? == 0) {
                                  ok($stdout =~ /^Result: PASS/ms);
                                  if($stdout =~ /Files=1, Tests=(\d+)/ms) {
                                          diag("$1 tests run");
                                  }
                          } else {
                                  diag("$filepath: STDOUT:\n$stdout");
                                  diag($stderr) if(length($stderr));
                                  diag("$filepath Failed");
                                  last;
                          }
                          diag($stderr) if(length($stderr));
                  }
          }
          closedir($dh);
    }

    done_testing();

## Property-Based Testing with Transforms

The generator can create property-based tests using [Test::LectroTest](https://metacpan.org/pod/Test%3A%3ALectroTest) when the
`properties` configuration option is enabled.
This provides more comprehensive
testing by automatically generating thousands of test cases and verifying that
mathematical properties hold across all inputs.

### Basic Property-Based Transform Example

Here's a complete example testing the `abs` builtin function:

**t/conf/abs.yml**:

    ---
    module: builtin
    function: abs

    config:
      test_undef: no
      test_empty: no
      test_nuls: no
      properties:
        enable: true
        trials: 1000

    input:
      number:
        type: number
        position: 0

    output:
      type: number
      min: 0

    transforms:
      positive:
        input:
          number:
            type: number
            min: 0
        output:
          type: number
          min: 0

      negative:
        input:
          number:
            type: number
            max: 0
        output:
          type: number
          min: 0

This configuration:

- Enables property-based testing with 1000 trials per property
- Defines two transforms: one for positive numbers, one for negative
- Automatically generates properties that verify `abs()` always returns non-negative numbers

Generate the test:

    fuzz-harness-generator t/conf/abs.yml > t/abs_property.t

The generated test will include:

- Traditional edge-case tests for boundary conditions
- Random fuzzing with 30 iterations (or as configured)
- Property-based tests that verify the transforms with 1000 trials each

### What Properties Are Tested?

The generator automatically detects and tests these properties based on your transform specifications:

- **Range constraints** - If output has `min` or `max`, verifies results stay within bounds
- **Type preservation** - Ensures numeric inputs produce numeric outputs
- **Definedness** - Verifies the function doesn't return `undef` unexpectedly
- **Specific values** - If output specifies a `value`, checks exact equality

For the `abs` example above, the generated properties verify:

    # For the "positive" transform:
    - Given a positive number, abs() returns >= 0
    - The result is a valid number
    - The result is defined

    # For the "negative" transform:
    - Given a negative number, abs() returns >= 0
    - The result is a valid number
    - The result is defined

### Advanced Example: String Normalization

Here's a more complex example testing a string normalization function:

**t/conf/normalize.yml**:

    ---
    module: Text::Processor
    function: normalize_whitespace

    config:
      properties:
        enable: true
        trials: 500

    input:
      text:
        type: string
        min: 0
        max: 1000
        position: 0

    output:
      type: string
      min: 0
      max: 1000

    transforms:
      empty_preserved:
        input:
          text:
            type: string
            value: ""
        output:
          type: string
          value: ""

      single_space:
        input:
          text:
            type: string
            min: 1
            matches: '^\S+(\s+\S+)*$'
        output:
          type: string
          matches: '^\S+( \S+)*$'

      length_bounded:
        input:
          text:
            type: string
            min: 1
            max: 100
        output:
          type: string
          min: 1
          max: 100

This tests that the normalization function:

- Preserves empty strings (`empty_preserved` transform)
- Collapses multiple spaces into single spaces (`single_space` transform)
- Maintains length constraints (`length_bounded` transform)

### Interpreting Property Test Results

When property-based tests run, you'll see output like:

    ok 123 - negative property holds (1000 trials)
    ok 124 - positive property holds (1000 trials)

If a property fails, Test::LectroTest will attempt to find the minimal failing
case and display it:

    not ok 123 - positive property holds (47 trials)
    # Property failed
    # Reason: counterexample found

This helps you quickly identify edge cases that your function doesn't handle correctly.

### Configuration Options for Property-Based Testing

In the `config` section:

    config:
      properties:
        enable: true     # Enable property-based testing (default: false)
        trials: 1000     # Number of test cases per property (default: 1000)

You can also disable traditional fuzzing and only use property-based tests:

    config:
      properties:
        enable: true
        trials: 5000

    iterations: 0  # Disable random fuzzing, use only property tests

### When to Use Property-Based Testing

Property-based testing with transforms is particularly useful for:

- Mathematical functions (`abs`, `sqrt`, `min`, `max`, etc.)
- Data transformations (encoding, normalization, sanitization)
- Parsers and formatters
- Functions with clear input-output relationships
- Code that should satisfy mathematical properties (commutativity, associativity, idempotence)

### Requirements

Property-based testing requires [Test::LectroTest](https://metacpan.org/pod/Test%3A%3ALectroTest) to be installed:

    cpanm Test::LectroTest

If not installed, the generated tests will automatically skip the property-based
portion with a message.

### Testing Email Validation

    ---
    module: Email::Valid
    function: rfc822

    config:
      properties:
        enable: true
        trials: 200
      close_stdin: true
      test_undef: no
      test_empty: no
      test_nuls: no

    input:
      email:
        type: string
        semantic: email
        position: 0

    output:
      type: boolean

    transforms:
      valid_emails:
        input:
          email:
            type: string
            semantic: email
        output:
          type: boolean

This generates 200 realistic email addresses for testing, rather than random strings.

### Combining Semantic with Regex

You can combine semantic generators with regex validation:

    input:
      corporate_email:
        type: string
        semantic: email
        matches: '@company\.com$'

The semantic generator creates realistic emails, and the regex ensures they match your domain.

### Custom Properties for Transforms

You can define additional properties that should hold for your transforms beyond
the automatically detected ones.

#### Using Built-in Properties

    transforms:
      positive:
        input:
          number:
            type: number
            min: 0
        output:
          type: number
          min: 0
        properties:
          - idempotent       # f(f(x)) == f(x)
          - non_negative     # result >= 0
          - positive         # result > 0

Available built-in properties:

- `idempotent` - Function is idempotent: f(f(x)) == f(x)
- `non_negative` - Result is always >= 0
- `positive` - Result is always > 0
- `non_empty` - String result is never empty
- `length_preserved` - Output length equals input length
- `uppercase` - Result is all uppercase
- `lowercase` - Result is all lowercase
- `trimmed` - No leading/trailing whitespace
- `sorted_ascending` - Array is sorted ascending
- `sorted_descending` - Array is sorted descending
- `unique_elements` - Array has no duplicates
- `preserves_keys` - Hash has same keys as input

#### Custom Property Code

Custom properties allows the definition additional invariants and relationships that should hold for their transforms,
beyond what's auto-detected.
For example:

- Idempotence: f(f(x)) == f(x)
- Commutativity: f(x, y) == f(y, x)
- Associativity: f(f(x, y), z) == f(x, f(y, z))
- Inverse relationships: decode(encode(x)) == x
- Domain-specific invariants: Custom business logic

Define your own properties with custom Perl code:

    transforms:
      normalize:
        input:
          text:
            type: string
        output:
          type: string
        properties:
          - name: single_spaces
            description: "No multiple consecutive spaces"
            code: $result !~ /  /

          - name: no_leading_space
            description: "No space at start"
            code: $result !~ /^\s/

          - name: reversible
            description: "Can be reversed back"
            code: length($result) == length($text)

The code has access to:

- `$result` - The function's return value
- Input variables - All input parameters (e.g., `$text`, `$number`)
- The function itself - Can call it again for idempotence checks

#### Combining Auto-detected and Custom Properties

The generator automatically detects properties from your output spec, and adds
your custom properties:

    transforms:
      sanitize:
        input:
          html:
            type: string
        output:
          type: string
          min: 0              # Auto-detects: defined, min_length >= 0
          max: 10000
        properties:           # Additional custom checks:
          - name: no_scripts
            code: $result !~ /<script/i
          - name: no_iframes
            code: $result !~ /<iframe/i

## GENERATED OUTPUT

The generated test:

- Seeds RND (if configured) for reproducible fuzz runs
- Uses edge cases (per-field and per-type) with configurable probability
- Runs `$iterations` fuzz cases plus appended edge-case runs
- Validates inputs with Params::Get / Params::Validate::Strict
- Validates outputs with [Return::Set](https://metacpan.org/pod/Return%3A%3ASet)
- Runs static `is(... )` corpus tests from Perl and/or YAML corpus
- Runs [Test::LectroTest](https://metacpan.org/pod/Test%3A%3ALectroTest) tests

# METHODS

## generate

    generate($schema_file, $test_file)

Takes a schema file and produces a test file (or STDOUT).

## render\_fallback

Render any Perl value into a compact Perl source-code string using
[Data::Dumper](https://metacpan.org/pod/Data%3A%3ADumper). Used as a catch-all when no more specific renderer
applies.

    my $code = render_fallback({ key => 'value' });
    # returns: "{'key' => 'value'}"

### Arguments

- `$v`

    Any Perl value, including undef, scalars, refs, and blessed objects.

### Returns

A string of Perl source code that reproduces the value when evaluated.
Returns the string `'undef'` when `$v` is undef.

### Side effects

Temporarily sets `$Data::Dumper::Terse` and `$Data::Dumper::Indent`
to produce compact single-line output. Both are restored on return via
`local`.

### Notes

The output is always a single line with no trailing newline. Suitable
for embedding in generated test code where readability is secondary to
correctness.

### API specification

#### input

    { v => { type => SCALAR|REF, optional => 1 } }

#### output

    { type => SCALAR }

## render\_hash

Render a two-level hashref (parameter name => spec hashref) into Perl
source code suitable for embedding in a generated test file as the
input specification passed to [Params::Validate::Strict](https://metacpan.org/pod/Params%3A%3AValidate%3A%3AStrict).

    my $code = render_hash(\%input);

### Arguments

- `$href`

    A hashref whose values are themselves hashrefs containing field
    specifications. Keys whose values are not hashrefs are skipped with
    a warning.

### Returns

A string of comma-separated Perl source-code lines, one per key, of
the form:

    'key' => { subkey => value, ... }

Returns an empty string if `$href` is undef, empty, or not a hashref.

### Side effects

None. Does not modify `$href`.

### Notes

The `matches` and `nomatch` sub-keys are treated specially — their
values are compiled to `Regexp` objects via `eval { qr/.../ }` and
then rendered using `perl_quote` so they appear as `qr{...}` in the
generated test. This prevents unmatched bracket characters in the
pattern from causing compilation failures.

Other sub-keys are rendered via `perl_quote`.

### API specification

#### input

    { href => { type => HASHREF, optional => 1 } }

#### output

    { type => SCALAR }

## render\_args\_hash

Render a flat hashref into a Perl source-code argument list of the
form `'key' =` value, ...>, suitable for embedding in a function call
in a generated test file.

    my $code = render_args_hash({ type => 'string', min => 1 });
    # returns: "'min' => 1, 'type' => 'string'"

### Arguments

- `$href`

    A flat hashref of key-value pairs. Values may be scalars, arrayrefs,
    or Regexp objects — all are handled by `perl_quote`.

### Returns

A comma-separated string of `key =` value> pairs sorted by key.
Returns an empty string if `$href` is undef, empty, or not a hashref.

### Side effects

None.

### Notes

Keys and values are both rendered via `perl_quote`. In particular,
`Regexp` values are rendered as `qr{...}` which is correct for
[Params::Validate::Strict](https://metacpan.org/pod/Params%3A%3AValidate%3A%3AStrict) and [Return::Set](https://metacpan.org/pod/Return%3A%3ASet) schema arguments in
the generated test.

### API specification

#### input

    { href => { type => HASHREF, optional => 1 } }

#### output

    { type => SCALAR }

## render\_arrayref\_map

Render a hashref whose values are arrayrefs into a Perl source-code
fragment suitable for use as a hash literal in a generated test file.

    my $code = render_arrayref_map({ name => ['', 'a' x 100] });

### Arguments

- `$href`

    A hashref whose values are arrayrefs. Keys whose values are not
    arrayrefs are silently skipped.

### Returns

A comma-separated string of `'key' =` \[ val, ... \]> entries, one per
qualifying key, sorted alphabetically. Returns the string `'()'` if
`$href` is undef, empty, or not a hashref — this produces an empty
hash assignment in the generated test rather than a syntax error.

### Side effects

None.

### Notes

Array element values are rendered via `perl_quote` which handles
scalars, arrayrefs, and Regexp objects. Non-arrayref values are
skipped without warning — this is intentional since callers may pass
mixed-value hashes and only want the arrayref entries rendered.

### API specification

#### input

    { href => { type => HASHREF, optional => 1 } }

#### output

    { type => SCALAR }

# NOTES

`seed` and `iterations` really should be within `config`.

# SEE ALSO

- [Test Coverage Report](https://nigelhorne.github.io/App-Test-Generator/coverage/)
- [App::Test::Generator::Template](https://metacpan.org/pod/App%3A%3ATest%3A%3AGenerator%3A%3ATemplate) - Template of the file of tests created by `App::Test::Generator`
- [App::Test::Generator::SchemaExtractor](https://metacpan.org/pod/App%3A%3ATest%3A%3AGenerator%3A%3ASchemaExtractor) - Create schemas from Perl programs
- [Params::Validate::Strict](https://metacpan.org/pod/Params%3A%3AValidate%3A%3AStrict): Schema Definition
- [Params::Get](https://metacpan.org/pod/Params%3A%3AGet): Input validation
- [Return::Set](https://metacpan.org/pod/Return%3A%3ASet): Output validation
- [Test::LectroTest](https://metacpan.org/pod/Test%3A%3ALectroTest)
- [Test::Most](https://metacpan.org/pod/Test%3A%3AMost)
- [YAML::XS](https://metacpan.org/pod/YAML%3A%3AXS)

# AUTHOR

Nigel Horne, `<njh at nigelhorne.com>`

Portions of this module's initial design and documentation were created with the
assistance of AI.

# SUPPORT

This module is provided as-is without any warranty.

You can find documentation for this module with the perldoc command.

    perldoc App::Test::Generator

You can also look for information at:

- MetaCPAN

    [https://metacpan.org/release/App-Test-Generator](https://metacpan.org/release/App-Test-Generator)

- GitHub

    [https://github.com/nigelhorne/App-Test-Generator](https://github.com/nigelhorne/App-Test-Generator)

- CPANTS

    [http://cpants.cpanauthors.org/dist/App-Test-Generator](http://cpants.cpanauthors.org/dist/App-Test-Generator)

- CPAN Testers' Matrix

    [http://matrix.cpantesters.org/?dist=App-Test-Generator](http://matrix.cpantesters.org/?dist=App-Test-Generator)

- CPAN Testers Dependencies

    [http://deps.cpantesters.org/?module=App::Test::Generator](http://deps.cpantesters.org/?module=App::Test::Generator)

# LICENCE AND COPYRIGHT

Copyright 2025-2026 Nigel Horne.

Usage is subject to the terms of GPL2.
If you use it,
please let me know.
