-
-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow for using python code from a separate Python module inside the specification #301
Comments
Hi, I would like to work on this. I'm also seeing the discussion happening on #51, so should the solution just expect to be parsing a reference to a function (e.g. Looking at the examples, accessing some of the response fields seems to be a key feature, so just a name wouldn't cut it. If the goal is only to be able to switch from python code to 'I want to call this function, no real python code involved', maybe there needs to be a symbol that tells the evaluator that this needs to be processed differently, such as a prefix or a different syntax, e.g. |
@zebralt thanks! I've assigned you to the ticket. About your point, I agree with you, the python code should have a context. I believe it should have the same context as it would have if it was only an expression as it is today. I mean, the context should be the same, using About the #51, I believe we still want the ability to have only an expression. This is specially good for people that are not fluent in Python. Just the way we validate it could be different, and not an eval. It would be nice to hear more thoughts here: @barbosa @flaviotruzzi @kelvins @marcuxyz, any comments? What do you think is the best way to have this feature? |
I'll give it a thought, but I think @zebralt is going in the right direction, I think Jinja use something similar. You should be able to have a context and patch the globals/locals when executing that code so it is sandboxed. It all depend on how much control and safety one would want. https://github.com/pallets/jinja/blob/9b718ed3d26fc2f80ab58f3249b517a65db9cc7b/src/jinja2/environment.py#L132 |
Although ScanAPI uses Python eval to evaluate Python expressions, it is a general tool that can be used for any web API, regardless of the programming language (as far as I understand it). With this feature, the tool will be more "coupled" to Python projects, since the Python support will be extended outside the One suggestion I can think of is to use the same pattern Gunicorn uses to define the app module, for example: In this case, we probably need to change the way the evaluation is performed. Another idea that I had, actually a workaround to avoid changing the way we perform evaluation, is to create a new modules:
test_module: testing.test # alias: module path
endpoints:
- name: pokemon
path: pokemon
requests:
- name: list_all
method: get
tests:
- name: status_code_is_200
assert: ${{ test_module.status_is_200(response) }} Then, internally we would need to dynamically import these modules using the alias and perform the Python >>> import importlib
>>> modules = {'test_module': 'testing.test'}
>>> for module_alias, module_path in modules.items():
>>> setattr(self, module_alias, importlib.import_module(module_path)) # inside a class
>>> eval('test_module.status_is_200(response)') Probably we would need to include a mechanism to avoid accidentally overriding attributes from the class itself. |
This is what I was thinking of when I spoke of explicitly importing modules: as part of the spec. You would then propagate this For now, I kept my work self-contained as a new
Yes, a separate syntax/feature might be the way to go for this current issue, especially if you want to retain backward compatibility with existing specs. So far I've been working on the custom syntax I proposed (exprs starting with a
Thanks for sharing that :D I wrote a similar function to fetch a module by name, but the parsing is interesting too; I was doing my own parsing of a In summarySandboxing seems like the optimal solution to the problem, but I'm not sure I can build an efficient solution for a problem of this magnitude in the span of time left for Hacktoberfest. Still I see there are off-the-shelf solutions like https://github.com/zopefoundation/RestrictedPython, so it may be worth giving a shot. This might just be a drop-in replacement for Or I can stay within the scope of the issue (some of this I have already done):
|
Hey, first of all, thank you for the awesome project. I'd like to know what is the status on this issue? Can I help with this issue? I might be able to contribute with some code, but I'd need some guidance. |
Hi @jlugao! Thank you very much 💜 This is not a simple issue, but it is a really nice one. It would be awesome to have it implemented. There is an open PR from last year that started the development: #324 We would need to:
If you want, I can tackle the number 1, since I have more familiar with the changes. |
@camilamaia that would be awesome. I would have no idea what to do on the rebase. But I sure can tackle the number 2. |
@jlugao Awesome! Number one is done ✅ #324 (comment) |
Closing this issue for now since there was no recent activity 🧹 |
Allow for using python code from a separate Python module inside the ScanAPI specification.
Example:
File tests.py
File scanapi.yaml
The text was updated successfully, but these errors were encountered: