Unclear about usage of cfn test with hooks

[jtaylor100]

I noticed that with #861 there were some test changes included for using cfn test with hooks.

See especially https://github.com/aws-cloudformation/cloudformation-cli/pull/861/files#diff-5a3cc3352be9423f5244d82f1ea97434ab5e7621366e2a589b0165b661c082aeR242

When trying to test on a hooks project generated by cfn init, as described on Testing resources types I find sam lambda start-local fails due to no template file being included. Without this command succeeding, cfn test contract tests fail with:

Validating your hook specification...
=================================================================================== test session starts ====================================================================================
platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-1.0.0 -- /usr/local/opt/[email protected]/bin/python3.9
cachedir: .pytest_cache
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('<HOOK_PROJ_DIR>/.hypothesis/examples')
rootdir: /private/var/folders/x_/pks7qs6d2nn3y2m_3z1c30qm0000gn/T, configfile: pytest_zjn9b1h7.ini
plugins: localserver-0.5.1.post0, random-order-1.0.4, hypothesis-6.39.3
collected 21 items / 16 deselected / 5 selected                                                                                                                                            

../../../../../../private/var/folders/x_/pks7qs6d2nn3y2m_3z1c30qm0000gn/T/hook/handler_pre_create.py::contract_pre_create_success FAILED                                             [ 20%]
../../../../../../private/var/folders/x_/pks7qs6d2nn3y2m_3z1c30qm0000gn/T/hook/handler_pre_create.py::contract_pre_create_failed FAILED                                              [ 40%]
../../../../../../private/var/folders/x_/pks7qs6d2nn3y2m_3z1c30qm0000gn/T/hook/handler_pre_update.py::contract_pre_update_success FAILED                                             [ 60%]
../../../../../../private/var/folders/x_/pks7qs6d2nn3y2m_3z1c30qm0000gn/T/hook/handler_pre_update.py::contract_pre_update_failed FAILED                                              [ 80%]
../../../../../../private/var/folders/x_/pks7qs6d2nn3y2m_3z1c30qm0000gn/T/resource/handler_misc.py::contract_check_asserts_work PASSED                                               [100%]

The contract tests that failed raised this exception:

E   botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://127.0.0.1:3001/2015-03-31/functions/TypeFunction/invocations"

I tried generating an empty template, just to check if having sam local start-lambda successfully running was enough, but cfn test raised this exception instead with the same test failures:

E           botocore.exceptions.ClientError: An error occurred (ResourceNotFound) when calling the Invoke operation: Function not found: arn:aws:lambda:us-west-2:012345678901:function:TypeFunction

---

It looks like cfn test is expecting a template to be included in the project that contains a definition for TypeFunction, but this is not included when generating a hooks project with cfn init.

Where am I going wrong in trying to use cfn test with my generated hooks project? There does not seem to be any docs written on this.

Does it involve writing a ~/.cfn-cli/typeConfigiration.json file as mentioned in the README.md? Are there also docs about the format of this file anywhere too or an example?

[wkexiang]

Hooks auto generates templete.yml for your local lambda to work. This file exists at project root. For example, if you run cfn init within ~/myhook, you will find that file in that directory. You need to run sam local start-lambda from that location. Please refer to Hooks Workshop documentation for detailed step by step instructions on testing your contract tests for Hooks locally. https://catalog.us-east-1.prod.workshops.aws/workshops/f09fd78b-ef8a-4a9d-9d2b-f31a3e6ca956/en-US/java/contracttesting

[cmgorton]

Hey @jtaylor100. We are working on answering and cleaning up questions.

If this is still an unanswered question for you, please feel free to reopen this issue so we can investigate it further.

You can also ask questions in our AWS CloudFormation Discord channel for a more prompt response.