From 600ab45c8aebb131544ec1442e6538016f4be188 Mon Sep 17 00:00:00 2001 From: Michael Howitz Date: Wed, 8 May 2019 13:55:22 +0200 Subject: [PATCH 1/3] Review documentation: * delete no longer needed parts * move and fix the rest --- README.rst | 2 +- docs/RestrictedPython3/index.rst | 39 ----- docs/RestrictedPython4/index.rst | 89 ----------- docs/api/index.rst | 22 --- docs/basics/index.rst | 48 ------ docs/call.txt | 6 - docs/conf.py | 4 +- docs/dep.txt | 70 --------- docs/idea.rst | 27 ++-- docs/index.rst | 12 +- docs/notes.rst | 98 ------------ docs/old_README.rst | 229 ---------------------------- docs/roadmap/index.rst | 4 +- docs/upgrade/ast/python2_6.ast | 142 ----------------- docs/upgrade/ast/python2_7.ast | 144 ----------------- docs/upgrade/ast/python3_0.ast | 147 ------------------ docs/upgrade/ast/python3_1.ast | 149 ------------------ docs/upgrade/ast/python3_2.ast | 146 ------------------ docs/upgrade/ast/python3_3.ast | 155 ------------------- docs/upgrade/ast/python3_4.ast | 156 ------------------- docs/upgrade/ast/python3_5.ast | 152 ------------------ docs/upgrade/ast/python3_6.ast | 169 -------------------- docs/upgrade/index.rst | 83 ---------- docs/upgrade_dependencies/index.rst | 19 +-- docs/usage/api.rst | 89 +++++------ docs/usage/basic_usage.rst | 2 +- docs/usage/framework_usage.rst | 29 ++-- docs/usage/index.rst | 10 +- docs/usage/policy.rst | 217 ++++++++++++++++++++++++-- 29 files changed, 288 insertions(+), 2171 deletions(-) delete mode 100644 docs/RestrictedPython3/index.rst delete mode 100644 docs/RestrictedPython4/index.rst delete mode 100644 docs/api/index.rst delete mode 100644 docs/basics/index.rst delete mode 100644 docs/call.txt delete mode 100644 docs/dep.txt delete mode 100644 docs/notes.rst delete mode 100644 docs/old_README.rst delete mode 100644 docs/upgrade/ast/python2_6.ast delete mode 100644 docs/upgrade/ast/python2_7.ast delete mode 100644 docs/upgrade/ast/python3_0.ast delete mode 100644 docs/upgrade/ast/python3_1.ast delete mode 100644 docs/upgrade/ast/python3_2.ast delete mode 100644 docs/upgrade/ast/python3_3.ast delete mode 100644 docs/upgrade/ast/python3_4.ast delete mode 100644 docs/upgrade/ast/python3_5.ast delete mode 100644 docs/upgrade/ast/python3_6.ast delete mode 100644 docs/upgrade/index.rst diff --git a/README.rst b/README.rst index 43db577..f09e46c 100644 --- a/README.rst +++ b/README.rst @@ -11,7 +11,7 @@ RestrictedPython is not a sandbox system or a secured environment, but it helps RestrictedPython only supports CPython. It does _not_ support PyPy and other Python implementations as it cannot provide its restrictions there. -For full documentation please see http://restrictedpython.readthedocs.io/ or the local ``docs/index``. +For full documentation please see http://restrictedpython.readthedocs.io/. Example ======= diff --git a/docs/RestrictedPython3/index.rst b/docs/RestrictedPython3/index.rst deleted file mode 100644 index 508690d..0000000 --- a/docs/RestrictedPython3/index.rst +++ /dev/null @@ -1,39 +0,0 @@ -RestrictedPython 3.6.x and before -================================= - -Technical foundation of RestrictedPython -........................................ - -RestrictedPython is based on the Python 2 only standard library module ``compiler`` (https://docs.python.org/2.7/library/compiler.html). -RestrictedPython based on the - -* ``compiler.ast`` -* ``compiler.parse`` -* ``compiler.pycodegen`` - -With Python 2.6 the compiler module with all its sub modules has been declared deprecated with no direct upgrade Path or recommendations for a replacement. - -Version Support of RestrictedPython 3.6.x -......................................... - -RestrictedPython 3.6.x aims on supporting Python versions: - -* 2.0 -* 2.1 -* 2.2 -* 2.3 -* 2.4 -* 2.5 -* 2.6 -* 2.7 - -Even if the README claims that Compatibility Support is form Python 2.3 - 2.7 I found some Code in RestrictedPython and related Packages which test if Python 1 is used. - -Due to this approach to support all Python 2 Versions the code uses only statements that are compatible with all of those versions. - -So old style classes and new style classes are mixed, - -The following language elements are statements and not functions: - -* exec -* print diff --git a/docs/RestrictedPython4/index.rst b/docs/RestrictedPython4/index.rst deleted file mode 100644 index dcd17f7..0000000 --- a/docs/RestrictedPython4/index.rst +++ /dev/null @@ -1,89 +0,0 @@ -RestrictedPython 4+ -=================== - -RestrictedPython 4 is a complete rewrite for Python 3 compatibility. - -Goals for a rewrite -------------------- - -RestrictedPython is a core dependency for the Zope2 application server and therefore for the content management system Plone. -The Zope & Plone community want to continue their projects and as Python 2 will reach its end-of-life by 2020, to be replaced by Python 3. -Zope and Plone should become Python 3 compatible. - -One of the core features of Zope 2 and therefore Plone is the possibility to implement and modify Python scripts and templates through the web (TTW) without harming the application or server itself. - -As Python is a `Turing complete `_ programming language programmers don't have any limitation and could potentially harm the Application and Server itself. - -RestrictedPython and AccessControl aims on this topic to provide a reduced subset of the Python Programming language, where all functions that could harm the system are permitted by default. - -Targeted Versions to support ----------------------------- - -For the RestrictedPython 4 update we aim to support only current Python -versions (the ones that will have active `security support `_ after this update -will be completed): - -* 2.7 -* 3.5 -* 3.6 -* 3.7 -* PyPy2.7 - -We explicitly excluded Python 3.3 and PyPy3 (which is based on the Python 3.3 specification) as the changes in Python 3.4 are significant and the Python 3.3 is nearing the end of its supported lifetime. - -Dependencies ------------- - -The following packages / modules have hard dependencies on RestrictedPython: - -* AccessControl --> -* zope.untrustedpython --> SelectCompiler -* DocumentTemplate --> -* Products.PageTemplates --> -* Products.PythonScripts --> -* Products.PluginIndexes --> -* five.pt (wrapping some functions and protection for Chameleon) --> - -Additionally the following add ons have dependencies on RestrictedPython - -* None - -How RestrictedPython 4+ works internally ----------------------------------------- - -RestrictedPython's core functions are split over several files: - -* __init__.py --> It exports the API directly in the ``RestrictedPython`` namespace. It should be not necessary to import from any other module inside the package. -* compile.py --> It contains the ``compile_restricted`` functions where internally ``_compile_restricted_mode`` is the important one -* transformer.py --> Home of the ``RestrictingNodeTransformer`` - -``RestrictingNodeTransformer`` -.............................. - -The ``RestrictingNodeTransformer`` is one of the core elements of RestrictedPython, it provides the base policy used by itself. - -``RestrictingNodeTransformer`` is a subclass of a ``NodeTransformer`` which has as set of ``visit_`` methods and a ``generic_visit`` method. - -``generic_visit`` is a predefined method of any ``NodeVisitor`` which sequential visit all sub nodes, in RestrictedPython this behavior is overwritten to always call a new internal method ``not_allowed(node)``. -This results in a implicit whitelisting of all allowed AST elements. -Any possible new introduced AST element in Python (new language element) will implicit be blocked and not allowed in RestrictedPython. - -So if new elements should be introduced an explicit ``visit_`` is necessary. - - -``_compile_restricted_mode`` -............................ - -``_compile_restricted_mode`` is an internal method that does the whole mapping against the used policy and compiles provided source code, with respecting the mode. -It is wrapped by the explicit functions: - -* ``compile_restricted_exec`` -* ``compile_restricted_eval`` -* ``compile_restricted_single`` -* ``compile_restricted_function`` - -They are still exposed as those are the nominal used API. - -For advanced usage this function is interesting as it is the point where the policy came into play. -If ``policy`` is ``None`` it just call the Python builtin ``compile`` method. -Else it parse the provided Python source code into an ``ast.AST`` and let it check and transform by the provided policy. diff --git a/docs/api/index.rst b/docs/api/index.rst deleted file mode 100644 index cdcae19..0000000 --- a/docs/api/index.rst +++ /dev/null @@ -1,22 +0,0 @@ -API of RestrictedPython 4.0 -=========================== - -.. code-block:: python - - compile_restricted(source, filename, mode [, flags [, dont_inherit]]) - -.. code-block:: python - - compile_restricted_exec(source, filename, mode [, flags [, dont_inherit [, policy]]]) - -.. code-block:: python - - compile_restricted_eval(source, filename, mode [, flags [, dont_inherit [, policy]]]) - -.. code-block:: python - - compile_restricted_single(source, filename, mode [, flags [, dont_inherit [, policy]]]) - -.. code-block:: python - - compile_restricted_function(source, filename, mode [, flags [, dont_inherit [, policy]]]) diff --git a/docs/basics/index.rst b/docs/basics/index.rst deleted file mode 100644 index 82a217b..0000000 --- a/docs/basics/index.rst +++ /dev/null @@ -1,48 +0,0 @@ -Grundlagen von RestrictedPython und der Sicherheitskonzepte von Zope2 -===================================================================== - -Motivation für RestrictedPython -------------------------------- - -Python ist eine moderne und heute sehr beliebte Programmiersprache. -Viele Bereiche nutzen heute Python ganz selbstverständlich. -Waren am Anfang gerade Systemadministratoren die via Python-Skripte ihre Systeme pflegten, ist heute die PyData Community eine der größten Nutzergruppen. -Auch wird Python gerne als Lehrsprache verwendet. - -Ein Nutzungsbereich von Python unterscheidet sich fundamental: *Python-Web* bzw. *Applikations-Server die Fremdcode aufnehmen*. -Zope gehörte zu den ersten großen und erfolgreichen Python-Web-Projekten und hat sich mit als erster um dieses Thema gekümmert. - -Während in der klassischen Software-Entwicklung aber auch in der Modelierung und Analyse von Daten drei Aspekte relevant sind: - -* Verständlichkeit des Programms (--> Siehe PEP 20 "The Zen of Python" https://www.python.org/dev/peps/pep-0020/) -* die Effizienz der Programmiersprache und der Ausführungsumgebung -* Verfügbarkeit der Ausführungsumgebung - -ist ein grundlegender Aspekt, die Mächtigkeit der Programmiersprache, selten von Relevanz. -Dies liegt auch daran, dass alle gängigen Programmiersprachen die gleiche Mächtigkeit besitzten: Turing-Vollständig. -Die Theoretische Informatik kennt mehrere Stufen der Mächtigkeit einer Programmiersprache, diese bilden die Grundlage der Berechenbarkeitstheorie. -Für klassische Software-Entwicklung ist eine Turing-vollständige Programmiersprache entsprechend die richtige Wahl. - -In der klassischen Software-Welt gelten in der Regel folgende Bedingungen: - -* man bekommt eine fertige Software und führt diese aus (Beispiel: Betriebssysteme, Anwendungen und Frameworks) -* man schreibt eine Software / Skript -* man verarbeitet Daten zur Berechung und Visualisierung, ohne ein vollumfängliches Programm zu entwickeln (Beispiel: MatLab, Jupyter-Notebooks) - -Da hierbei erstmal keine Unterscheidung zwischen Open Source und Closed Source Software gemacht werden soll, da die relevante Frage eher eine Frage des Vertrauen ist. - -Die zentrale Frage ist: - - Vertraue ich der Software, bzw. den Entwicklern der Software und führe diese aus. - -Python ist eine Turing-vollständige Prgrammiersprache. -Somit haben Entwickler grundsätzlich erstmal keine Limitierungen beim programmieren. - -und können somit potentiell die Applikation und den Server selber schaden. - -RestrictedPython und AccessControl zielen auf diese Besonderheit und versuchen einen reduzierten Subset der Programmiersprache Python zur verfügung zu stellen. -Hierzu werden erstmal alle Funktionen die potentiel das System schaden können verboten. -Genauer gesagt muss jede Funktion, egal ob eine der Python ``__builtin__``-Funktionen, der Python Standard-Library oder beliebiger Zusatz-Modulen / (Python-Eggs) explizit freigegeben werden. -Wie sprechen hier von White-Listing. - -Damit dies funktioniert, muss neben der ``restricted_compile``-Funktion auch eine API für die explizite Freigabe von Modulen und Funktionen existieren. diff --git a/docs/call.txt b/docs/call.txt deleted file mode 100644 index b333f5e..0000000 --- a/docs/call.txt +++ /dev/null @@ -1,6 +0,0 @@ -collective.themefragments: /collective/themefragments/traversal.py:6:from RestrictedPython import compile_restricted_function -collective.themefragments: /collective/themefragments/traversal.py:29: r = compile_restricted_function(p, body, name, filename, globalize) -Products.PythonScripts: /Products/PythonScripts/PythonScript.py:45:from RestrictedPython import compile_restricted_function -Products.PythonScripts: /Products/PythonScripts/PythonScript.py:239: return compile_restricted_function(*args, **kw) -Zope2: /Products/PageTemplates/ZRPythonExpr.py:20:from RestrictedPython import compile_restricted_eval -Zope2: /Products/PageTemplates/ZRPythonExpr.py:36: code, err, warn, use = compile_restricted_eval(text, diff --git a/docs/conf.py b/docs/conf.py index cec05e2..6c70069 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -56,9 +56,9 @@ # built documents. # # The short X.Y version. -version = u'4.0.0.dev0' +version = u'4.0' # The full version, including alpha/beta/rc tags. -release = u'4.0.0.dev0' +release = u'4.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/dep.txt b/docs/dep.txt deleted file mode 100644 index 9860f60..0000000 --- a/docs/dep.txt +++ /dev/null @@ -1,70 +0,0 @@ -AccessControl --> /AccessControl/tests/testZopeGuards.py:471:class TestRestrictedPythonApply(GuardTestCase): -AccessControl --> /AccessControl/tests/testZopeGuards.py:517:# Given the high wall between AccessControl and RestrictedPython, I suppose -AccessControl --> /AccessControl/tests/testZopeGuards.py:601: from RestrictedPython.tests import verify -AccessControl --> /AccessControl/tests/testZopeGuards.py:631: from RestrictedPython.tests import verify -AccessControl --> /AccessControl/tests/testZopeGuards.py:658: from RestrictedPython.tests import verify -AccessControl --> /AccessControl/tests/testZopeGuards.py:680: from RestrictedPython.tests import verify -AccessControl --> /AccessControl/tests/testZopeGuards.py:702: from RestrictedPython.tests import verify -AccessControl --> /AccessControl/tests/testZopeGuards.py:722: from RestrictedPython.tests import verify -AccessControl --> /AccessControl/tests/testZopeGuards.py:748: from RestrictedPython import compile_restricted -AccessControl --> /AccessControl/tests/testZopeGuards.py:764: from RestrictedPython import compile_restricted -AccessControl --> /AccessControl/tests/testZopeGuards.py:872: TestRestrictedPythonApply, - -AccessControl --> /AccessControl/ZopeGuards.py:20:import RestrictedPython -AccessControl --> /AccessControl/ZopeGuards.py:21:from RestrictedPython.Guards import safe_builtins, full_write_guard -AccessControl --> /AccessControl/ZopeGuards.py:22:from RestrictedPython.Utilities import utility_builtins -AccessControl --> /AccessControl/ZopeGuards.py:23:from RestrictedPython.Eval import RestrictionCapableEval -AccessControl --> /AccessControl/ZopeGuards.py:540: '_print_': RestrictedPython.PrintCollector, - -collective.themefragments --> collective/themefragments/traversal.py:6:from RestrictedPython import compile_restricted_function - -DocumentTemplate --> DocumentTemplate/DT_Util.py:25:from RestrictedPython.Guards import safe_builtins -DocumentTemplate --> DocumentTemplate/DT_Util.py:26:from RestrictedPython.Utilities import utility_builtins -DocumentTemplate --> DocumentTemplate/DT_Util.py:27:from RestrictedPython.Eval import RestrictionCapableEval -DocumentTemplate --> DocumentTemplate/DT_Util.py:38: from RestrictedPython.Utilities import test -DocumentTemplate --> DocumentTemplate/DT_Util.py:79: from RestrictedPython.Limits import limited_builtins - -five.pt-2.2.3-py2.7.egg/five/pt/expressions.py:23:from RestrictedPython.RestrictionMutator import RestrictionMutator -five.pt-2.2.3-py2.7.egg/five/pt/expressions.py:24:from RestrictedPython.Utilities import utility_builtins -five.pt-2.2.3-py2.7.egg/five/pt/expressions.py:25:from RestrictedPython import MutatingWalker - -Products.CMFPlone-4.3.9-py2.7.egg/Products/CMFPlone/tests/testSecurity.py:32: def test_PT_allow_module_not_available_in_RestrictedPython_1(self): -Products.CMFPlone-4.3.9-py2.7.egg/Products/CMFPlone/tests/testSecurity.py:47: def test_PT_allow_module_not_available_in_RestrictedPython_2(self): -Products.CMFPlone-4.3.9-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:16:class RestrictedPythonTest(ZopeTestCase.ZopeTestCase): -Products.CMFPlone-4.3.9-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:38:class TestSecurityDeclarations(RestrictedPythonTest): -Products.CMFPlone-4.3.9-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:343:class TestAcquisitionMethods(RestrictedPythonTest): -Products.CMFPlone-4.3.9-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:432:class TestNavtreeSecurity(PloneTestCase.PloneTestCase, RestrictedPythonTest): - -Products.CMFPlone-5.0.2-py2.7.egg/Products/CMFPlone/tests/testSecurity.py:30: def test_PT_allow_module_not_available_in_RestrictedPython_1(self): -Products.CMFPlone-5.0.2-py2.7.egg/Products/CMFPlone/tests/testSecurity.py:45: def test_PT_allow_module_not_available_in_RestrictedPython_2(self): -Products.CMFPlone-5.0.2-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:15:class RestrictedPythonTest(TestCase): -Products.CMFPlone-5.0.2-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:49:class TestSecurityDeclarations(RestrictedPythonTest): -Products.CMFPlone-5.0.2-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:325:class TestAcquisitionMethods(RestrictedPythonTest): -Products.CMFPlone-5.0.2-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:400:class TestNavtreeSecurity(PloneTestCase.PloneTestCase, RestrictedPythonTest): - -Products.CMFPlone-5.1a1-py2.7.egg/Products/CMFPlone/tests/testSecurity.py:30: def test_PT_allow_module_not_available_in_RestrictedPython_1(self): -Products.CMFPlone-5.1a1-py2.7.egg/Products/CMFPlone/tests/testSecurity.py:45: def test_PT_allow_module_not_available_in_RestrictedPython_2(self): -Products.CMFPlone-5.1a1-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:15:class RestrictedPythonTest(TestCase): -Products.CMFPlone-5.1a1-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:49:class TestSecurityDeclarations(RestrictedPythonTest): -Products.CMFPlone-5.1a1-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:325:class TestAcquisitionMethods(RestrictedPythonTest): -Products.CMFPlone-5.1a1-py2.7.egg/Products/CMFPlone/tests/testSecurityDeclarations.py:400:class TestNavtreeSecurity(PloneTestCase.PloneTestCase, RestrictedPythonTest): - -Products.PythonScripts --> Products/PythonScripts/PythonScript.py:45:from RestrictedPython import compile_restricted_function -Products.PythonScripts --> Products/PythonScripts/tests/testPythonScript.py:18:from RestrictedPython.tests.verify import verify - -Products.ZCatalog --> /Products/PluginIndexes/TopicIndex/FilteredSet.py:19:from RestrictedPython.Eval import RestrictionCapableEval - -zope.security-3.7.4-py2.7-macosx-10.10-x86_64.egg/zope/security/untrustedpython/rcompile.py:23:import RestrictedPython.RCompile -zope.security-3.7.4-py2.7-macosx-10.10-x86_64.egg/zope/security/untrustedpython/rcompile.py:24:from RestrictedPython.SelectCompiler import ast, OP_ASSIGN, OP_DELETE, OP_APPLY -zope.security-3.7.4-py2.7-macosx-10.10-x86_64.egg/zope/security/untrustedpython/rcompile.py:33:class RExpression(RestrictedPython.RCompile.RestrictedCompileMode): -zope.security-3.7.4-py2.7-macosx-10.10-x86_64.egg/zope/security/untrustedpython/rcompile.py:39: RestrictedPython.RCompile.RestrictedCompileMode.__init__( -zope.security-3.7.4-py2.7-macosx-10.10-x86_64.egg/zope/security/untrustedpython/rcompile.txt:28:The implementation makes use of the `RestrictedPython` package, - -zope.untrustedpython --> rcompile.py:21:import RestrictedPython.RCompile -zope.untrustedpython --> rcompile.py:22:from RestrictedPython.SelectCompiler import ast -zope.untrustedpython --> rcompile.py:31:class RExpression(RestrictedPython.RCompile.RestrictedCompileMode): -zope.untrustedpython --> rcompile.py:37: RestrictedPython.RCompile.RestrictedCompileMode.__init__( - -* Products.PageTemplates --> ZRPythonExpr.py:15:Handler for Python expressions that uses the RestrictedPython package. -* Products.PageTemplates --> ZRPythonExpr.py:20:from RestrictedPython import compile_restricted_eval -* Products.PageTemplates --> ZRPythonExpr.py:32: # Unicode expression are not handled properly by RestrictedPython diff --git a/docs/idea.rst b/docs/idea.rst index d462a1d..8b9c8c3 100644 --- a/docs/idea.rst +++ b/docs/idea.rst @@ -1,12 +1,12 @@ -The Idea behind RestrictedPython +The idea behind RestrictedPython ================================ Python is a `Turing complete `_ programming language. To offer a Python interface for users in web context is a potential security risk. Web frameworks and Content Management Systems (CMS) want to offer their users as much extensibility as possible through the web (TTW). -This also means to have permissions to add functionality via a Python Script. +This also means to have permissions to add functionality via a Python script. -There should be additional preventive measures taken to ensure integrity of the application and the server itself, according to information security best practice and unrelated to Restricted Python. +There should be additional preventive measures taken to ensure integrity of the application and the server itself, according to information security best practice and unrelated to RestrictedPython. RestrictedPython defines a safe subset of the Python programming language. This is a common approach for securing a programming language. @@ -16,8 +16,7 @@ Defining a secure subset of the language involves restricting the `EBNF `_ / `Python 3 `_). -This method is defined as following: +This Python function is defined as following: .. code-block:: python @@ -44,14 +43,14 @@ For RestrictedPython this ``compile()`` method is replaced by: .. code-block:: python - compile_restricted(source, filename, mode [, flags [, dont_inherit]]) + RestrictedPython.compile_restricted(source, filename, mode [, flags [, dont_inherit]]) -The primary parameter ``source`` has to be a ASCII or ``unicode`` string (With Python 2.6 an additional option for source was added: ``ast.AST`` for :ref:`Code generation <_sec_code_generation>`). -Both methods either returns compiled byte code that the interpreter could execute or raise exceptions if the provided source code is invalid. +The primary parameter ``source`` has to be a string or ``ast.AST`` instance. +Both methods either return compiled byte code that the interpreter can execute or raise exceptions if the provided source code is invalid. As ``compile`` and ``compile_restricted`` just compile the provided source code to byte code it is not sufficient to sandbox the environment, as all calls to libraries are still available. -The two methods / Statements: +The two methods / statements: * ``exec`` / ``exec()`` * ``eval`` / ``eval()`` @@ -61,9 +60,9 @@ have two parameters: * ``globals`` * ``locals`` -which are a reference to the Python builtins. +which are references to the Python builtins. -By modifying and restricting the available modules, methods and constants from globals and locals we could limit the possible calls. +By modifying and restricting the available modules, methods and constants from ``globals`` and ``locals`` we can limit the possible calls. Additionally RestrictedPython offers a way to define a policy which allows developers to protect access to attributes. This works by defining a restricted version of: @@ -73,7 +72,7 @@ This works by defining a restricted version of: * ``setattr`` * ``import`` -Also RestrictedPython provides three predefined, limited versions of Python's own ``__builtins__``: +Also RestrictedPython provides three predefined, limited versions of Python's ``__builtins__``: * ``safe_builtins`` (by Guards.py) * ``limited_builtins`` (by Limits.py), which provides restricted sequence types @@ -83,4 +82,4 @@ One special shortcut: * ``safe_globals`` for ``{'__builtins__': safe_builtins}`` (by Guards.py) -Additional there exist guard functions to make attributes of Python objects immutable --> ``full_write_guard`` (write and delete protected) +Additional there exist guard functions to make attributes of Python objects immutable --> ``full_write_guard`` (write and delete protected). diff --git a/docs/index.rst b/docs/index.rst index b103fb4..19a6793 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -9,12 +9,13 @@ Welcome to RestrictedPython's documentation! ============================================ -.. include:: idea.rst +RestrictedPython is a tool that helps to define a subset of the Python language which allows to provide a program input into a trusted environment. +RestrictedPython is not a sandbox system or a secured environment, but it helps to define a trusted environment and execute untrusted code inside of it. Supported Python versions ========================= -RestrictedPython support CPython 2.7, 3.5, 3.6 and 3.7. +RestrictedPython supports CPython 2.7, 3.5, 3.6, 3.7 and 3.8. It does _not_ support PyPy or other alternative Python implementations. Contents @@ -24,13 +25,10 @@ Contents :maxdepth: 2 idea - basics/index + install/index usage/index - api/index + usage/api - RestrictedPython3/index - RestrictedPython4/index - upgrade/index upgrade_dependencies/index roadmap/index diff --git a/docs/notes.rst b/docs/notes.rst deleted file mode 100644 index 438122d..0000000 --- a/docs/notes.rst +++ /dev/null @@ -1,98 +0,0 @@ -How it works -============ - -*Caution:* This is old documentation from RestrictedPython 3 and before. -Information should be transferred and this file should be removed. - -Every time I see this code, I have to relearn it. These notes will -hopefully make this a little easier. :) - -- The important module is RCompile. The entry points are the - compile_restricted_* functions. - - + compile_restricted_function is used by Python scripts. - - + compile_restricted_eval is used by ZPT - - and by DTML indirectly through Eval.RestrictionCapableEval. - -- OK, so lets see how this works by following the logic of - compile_restricted_eval. - - - First, we create an RExpression, passing the source and a - "file name", to be used in tracebacks. - - Now, an RExpression is just: - - + a subclass of RestrictedCompileMode and Expression. - - Expression is a subclass of AbstractCompileMode that sets it's - mode to 'eval' and everided compile. Sigh. - - + RestrictedCompileMode is a subclass of AbstractCompileMode - that changes a bunch of things. :) These include compile, so we - can ignore the compile we got from Expression. It would have - been simpler to just set the dang mode in RExpression. Sigh. - - RestrictedCompileMode seem to be the interesting base class. I - assume it implements the interesting functionality. We'll see - below... - - - Next, we call compileAndTuplize. - - + This calls compile on the RExpression. It has an error - handler that does something that I hope I don't care about. :) - - + It then calls the genCode method on the RExpression. - This is boring, so we'll not worry about it. - - - The compile method provided by RestrictedCompileMode is - interesting. - - + First it calls _get_tree. - - * It uses compiler.parse to parse the source - - * it uses MutatingWalker.walk to mutate the tree using the - RestrictedCompileMode's 'rm' attr, which is a - RestrictionMutator. - - The RestrictionMutator has the recipies for mutating the parse - tree. (Note, for comparison, that Zope3's - zope.untrustedpython.rcompile module an alternative - RestrictionMutator that provides a much smaller set of - changes.) - - A mutator has visit method for different kinds of AST - nodes. These visit methods may mutate nodes or return new - nodes that replace the originally visited nodes. There is a - default visitor that visits a node's children and replaces the - children whose visitors returned new nodes. - - The walk function just calls the visitor for the root node of - the given tree. Note _get_tree ignores the walk return value, - thus assuming that the visitor for the root node doesn't - return a new node. This is a theoretical bug that we can - ignore. - - + Second, it generates the code. This too is boring. - -- So this seems simple enough. ;) When we want to add a check, we - need to update or add a visit function in RestrictionMutator. - - How does a visit function work. - - - First, we usually call walker.defaultVisitNode(node). This - transforms the node's child nodes. - - - Then we hack the node, or possibly return the node. To do this, we - have to know how the node works. - - - The hack often involved changing the code to call some checker - function. These have names like _name_. These are names that - would be illegal in the input source. - - If this is a new function, we have to provide it in - AccessControl.ZopeGuards._safe_globals. - -- Don't forget to add a test case to tests.before_and_after. diff --git a/docs/old_README.rst b/docs/old_README.rst deleted file mode 100644 index f2384f0..0000000 --- a/docs/old_README.rst +++ /dev/null @@ -1,229 +0,0 @@ -.. contents:: - -Overview -======== - -RestrictedPython provides a ``restricted_compile`` function that works -like the built-in ``compile`` function, except that it allows the -controlled and restricted execution of code: - -.. code-block:: pycon - - >>> src = ''' - ... def hello_world(): - ... return "Hello World!" - ... ''' - >>> from RestrictedPython import compile_restricted - >>> code = compile_restricted(src, '', 'exec') - -The resulting code can be executed using the ``exec`` built-in: - -.. code-block:: pycon - - >>> exec(code) - -As a result, the ``hello_world`` function is now available in the -global namespace: - -.. code-block:: pycon - - >>> hello_world() - 'Hello World!' - -Compatibility -============= - -This release of RestrictedPython is compatible with Python 2.7, 3.4, 3.5, 3.6. - -Implementing a policy -===================== - -RestrictedPython only provides the raw material for restricted execution. -To actually enforce any restrictions, you need to supply a policy -implementation by providing restricted versions of ``print``, -``getattr``, ``setattr``, ``import``, etc. These restricted -implementations are hooked up by providing a set of specially named -objects in the global dict that you use for execution of code. -Specifically: - -1. ``_print_`` is a callable object that returns a handler for print - statements. This handler must have a ``write()`` method that - accepts a single string argument, and must return a string when - called. ``RestrictedPython.PrintCollector.PrintCollector`` is a - suitable implementation. - -2. ``_write_`` is a guard function taking a single argument. If the - object passed to it may be written to, it should be returned, - otherwise the guard function should raise an exception. ``_write`` - is typically called on an object before a ``setattr`` operation. - -3. ``_getattr_`` and ``_getitem_`` are guard functions, each of which - takes two arguments. The first is the base object to be accessed, - while the second is the attribute name or item index that will be - read. The guard function should return the attribute or subitem, - or raise an exception. - RestrictedPython ships with a default implementation - for ``_getattr_`` which prevents the following actions: - - * accessing an attribute those name start with an underscore - * accessing the format method of strings as it is considered harmful. - -4. ``__import__`` is the normal Python import hook, and should be used - to control access to Python packages and modules. - -5. ``__builtins__`` is the normal Python builtins dictionary, which - should be weeded down to a set that cannot be used to get around - your restrictions. A usable "safe" set is - ``RestrictedPython.Guards.safe_builtins``. - -To help illustrate how this works under the covers, here's an example -function: - -.. code-block:: python - - def f(x): - x.foo = x.foo + x[0] - print x - return printed - -and (sort of) how it looks after restricted compilation: - -.. code-block:: python - - def f(x): - # Make local variables from globals. - _print = _print_() - _write = _write_ - _getattr = _getattr_ - _getitem = _getitem_ - - # Translation of f(x) above - _write(x).foo = _getattr(x, 'foo') + _getitem(x, 0) - print >>_print, x - return _print() - -Examples -======== - -``print`` ---------- - -To support the ``print`` statement in restricted code, we supply a -``_print_`` object (note that it's a *factory*, e.g. a class or a -callable, from which the restricted machinery will create the object): - -.. code-block:: pycon - - >>> from RestrictedPython.PrintCollector import PrintCollector - >>> _print_ = PrintCollector - >>> _getattr_ = getattr - - >>> src = ''' - ... print("Hello World!") - ... ''' - >>> code = compile_restricted(src, '', 'exec') - >>> exec(code) - -As you can see, the text doesn't appear on stdout. The print -collector collects it. We can have access to the text using the -``printed`` variable, though: - -.. code-block:: pycon - - >>> src = ''' - ... print("Hello World!") - ... result = printed - ... ''' - >>> code = compile_restricted(src, '', 'exec') - >>> exec(code) - - >>> result - 'Hello World!\n' - -Built-ins ---------- - -By supplying a different ``__builtins__`` dictionary, we can rule out -unsafe operations, such as opening files: - -.. code-block:: pycon - - >>> from RestrictedPython.Guards import safe_builtins - >>> restricted_globals = dict(__builtins__ = safe_builtins) - - >>> src = ''' - ... open('/etc/passwd') - ... ''' - >>> code = compile_restricted(src, '', 'exec') - >>> exec(code, restricted_globals) - Traceback (most recent call last): - ... - NameError: name 'open' is not defined - -Guards ------- - -Here's an example of a write guard that never lets restricted code -modify (assign, delete an attribute or item) except dictionaries and -lists: - -.. code-block:: pycon - - >>> from RestrictedPython.Guards import full_write_guard - >>> _write_ = full_write_guard - >>> _getattr_ = getattr - - >>> class BikeShed(object): - ... colour = 'green' - ... - >>> shed = BikeShed() - -Normally accessing attriutes works as expected, because we're using -the standard ``getattr`` function for the ``_getattr_`` guard: - -.. code-block:: pycon - - >>> src = ''' - ... print(shed.colour) - ... result = printed - ... ''' - >>> code = compile_restricted(src, '', 'exec') - >>> exec(code) - - >>> result - 'green\n' - -However, changing an attribute doesn't work: - -.. code-block:: pycon - - >>> src = ''' - ... shed.colour = 'red' - ... ''' - >>> code = compile_restricted(src, '', 'exec') - >>> exec(code) - Traceback (most recent call last): - ... - TypeError: attribute-less object (assign or del) - -As said, this particular write guard (``full_write_guard``) will allow -restricted code to modify lists and dictionaries: - -.. code-block:: pycon - - >>> fibonacci = [1, 1, 2, 3, 4] - >>> transl = dict(one=1, two=2, tres=3) - >>> src = ''' - ... # correct mistake in list - ... fibonacci[-1] = 5 - ... # one item doesn't belong - ... del transl['tres'] - ... ''' - >>> code = compile_restricted(src, '', 'exec') - >>> exec(code) - - >>> fibonacci - [1, 1, 2, 3, 5] - - >>> sorted(transl.keys()) - ['one', 'two'] diff --git a/docs/roadmap/index.rst b/docs/roadmap/index.rst index a854496..a483866 100644 --- a/docs/roadmap/index.rst +++ b/docs/roadmap/index.rst @@ -1,5 +1,5 @@ -Roadmap for RestrictedPython -============================ +Roadmap +======= RestrictedPython 4.0 -------------------- diff --git a/docs/upgrade/ast/python2_6.ast b/docs/upgrade/ast/python2_6.ast deleted file mode 100644 index 3ea12f8..0000000 --- a/docs/upgrade/ast/python2_6.ast +++ /dev/null @@ -1,142 +0,0 @@ --- Python 2.6 AST --- ASDL's five builtin types are identifier, int, string, object, bool - -module Python version "2.6" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, arguments args, - stmt* body, expr* decorator_list) - | ClassDef(identifier name, expr* bases, stmt* body, expr *decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- not sure if bool is allowed, can always use int - | Print(expr? dest, expr* values, bool nl) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(expr context_expr, expr? optional_vars, stmt* body) - - -- 'type' is a bad name - | Raise(expr? type, expr? inst, expr? tback) - | TryExcept(stmt* body, excepthandler* handlers, stmt* orelse) - | TryFinally(stmt* body, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier module, alias* names, int? level) - - -- Doesn't capture requirement that locals must be - -- defined if globals is - -- still supports use as a function! - | Exec(expr body, expr? globals, expr? locals) - - | Global(identifier* names) - | Expr(expr value) - | Pass | Break | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | ListComp(expr elt, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Yield(expr? value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords, - expr? starargs, expr? kwargs) - | Repr(expr value) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - -- other literals? bools? - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Ellipsis - | Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And - | Or - - operator = Add - | Sub - | Mult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - -- not sure what to call the first argument for raise and except - excepthandler = ExceptHandler(expr? type, expr? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (expr* args, identifier? vararg, - identifier? kwarg, expr* defaults) - - -- keyword arguments supplied to call - keyword = (identifier arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) -} diff --git a/docs/upgrade/ast/python2_7.ast b/docs/upgrade/ast/python2_7.ast deleted file mode 100644 index 899cdc5..0000000 --- a/docs/upgrade/ast/python2_7.ast +++ /dev/null @@ -1,144 +0,0 @@ --- Python 2.7 AST --- ASDL's five builtin types are identifier, int, string, object, bool - -module Python version "2.7" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, arguments args, - stmt* body, expr* decorator_list) - | ClassDef(identifier name, expr* bases, stmt* body, expr* decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- not sure if bool is allowed, can always use int - | Print(expr? dest, expr* values, bool nl) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(expr context_expr, expr? optional_vars, stmt* body) - - -- 'type' is a bad name - | Raise(expr? type, expr? inst, expr? tback) - | TryExcept(stmt* body, excepthandler* handlers, stmt* orelse) - | TryFinally(stmt* body, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier? module, alias* names, int? level) - - -- Doesn't capture requirement that locals must be - -- defined if globals is - -- still supports use as a function! - | Exec(expr body, expr? globals, expr? locals) - - | Global(identifier* names) - | Expr(expr value) - | Pass | Break | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Yield(expr? value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords, - expr? starargs, expr? kwargs) - | Repr(expr value) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - -- other literals? bools? - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Ellipsis - | Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And | Or - - operator = Add - | Sub - | Mult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - -- not sure what to call the first argument for raise and except - excepthandler = ExceptHandler(expr? type, expr? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (expr* args, identifier? vararg, - identifier? kwarg, expr* defaults) - - -- keyword arguments supplied to call - keyword = (identifier arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) -} diff --git a/docs/upgrade/ast/python3_0.ast b/docs/upgrade/ast/python3_0.ast deleted file mode 100644 index 44d6e18..0000000 --- a/docs/upgrade/ast/python3_0.ast +++ /dev/null @@ -1,147 +0,0 @@ --- Python 3.0 AST --- ASDL's four builtin types are identifier, int, string, object - -module Python version "3.0" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, arguments args, - stmt* body, expr* decorator_list, expr? returns) - | ClassDef(identifier name, - expr* bases, - keyword* keywords, - expr? starargs, - expr? kwargs, - stmt* body, - expr *decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(expr context_expr, expr? optional_vars, stmt* body) - - | Raise(expr? exc, expr? cause) - | TryExcept(stmt* body, excepthandler* handlers, stmt* orelse) - | TryFinally(stmt* body, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier module, alias* names, int? level) - - | Global(identifier* names) - | Nonlocal(identifier* names) - | Expr(expr value) - | Pass | Break | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Yield(expr? value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords, - expr? starargs, expr? kwargs) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - | Bytes(string s) - | Ellipsis - -- other literals? bools? - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Starred(expr value, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And - | Or - - operator = Add - | Sub - | Mult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - -- not sure what to call the first argument for raise and except - excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (arg* args, identifier? vararg, expr? varargannotation, - arg* kwonlyargs, identifier? kwarg, - expr? kwargannotation, expr* defaults, - expr* kw_defaults) - arg = (identifier arg, expr? annotation) - - -- keyword arguments supplied to call - keyword = (identifier arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) -} diff --git a/docs/upgrade/ast/python3_1.ast b/docs/upgrade/ast/python3_1.ast deleted file mode 100644 index a0bbfb6..0000000 --- a/docs/upgrade/ast/python3_1.ast +++ /dev/null @@ -1,149 +0,0 @@ --- Python 3.1 AST --- ASDL's four builtin types are identifier, int, string, object - -module Python version "3.1" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, arguments args, - stmt* body, expr* decorator_list, expr? returns) - | ClassDef(identifier name, - expr* bases, - keyword* keywords, - expr? starargs, - expr? kwargs, - stmt* body, - expr *decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(expr context_expr, expr? optional_vars, stmt* body) - - | Raise(expr? exc, expr? cause) - | TryExcept(stmt* body, excepthandler* handlers, stmt* orelse) - | TryFinally(stmt* body, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier module, alias* names, int? level) - - | Global(identifier* names) - | Nonlocal(identifier* names) - | Expr(expr value) - | Pass - | Break - | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Yield(expr? value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords, - expr? starargs, expr? kwargs) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - | Bytes(string s) - | Ellipsis - -- other literals? bools? - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Starred(expr value, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And - | Or - - operator = Add - | Sub - | Mult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - -- not sure what to call the first argument for raise and except - excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (arg* args, identifier? vararg, expr? varargannotation, - arg* kwonlyargs, identifier? kwarg, - expr? kwargannotation, expr* defaults, - expr* kw_defaults) - arg = (identifier arg, expr? annotation) - - -- keyword arguments supplied to call - keyword = (identifier arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) -} diff --git a/docs/upgrade/ast/python3_2.ast b/docs/upgrade/ast/python3_2.ast deleted file mode 100644 index 054d63e..0000000 --- a/docs/upgrade/ast/python3_2.ast +++ /dev/null @@ -1,146 +0,0 @@ --- Python 3.2 AST --- ASDL's four builtin types are identifier, int, string, object - -module Python version "3.2" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, - arguments args, - stmt* body, - expr* decorator_list, - expr? returns) - | ClassDef(identifier name, - expr* bases, - keyword* keywords, - expr? starargs, - expr? kwargs, - stmt* body, - expr* decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(expr context_expr, expr? optional_vars, stmt* body) - - | Raise(expr? exc, expr? cause) - | TryExcept(stmt* body, excepthandler* handlers, stmt* orelse) - | TryFinally(stmt* body, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier? module, alias* names, int? level) - - | Global(identifier* names) - | Nonlocal(identifier* names) - | Expr(expr value) - | Pass - | Break - | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Yield(expr? value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords, - expr? starargs, expr? kwargs) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - | Bytes(string s) - | Ellipsis - -- other literals? bools? - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Starred(expr value, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load | Store | Del | AugLoad | AugStore | Param - - slice = Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And | Or - - operator = Add - | Sub - | Mult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - -- not sure what to call the first argument for raise and except - excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (arg* args, identifier? vararg, expr? varargannotation, - arg* kwonlyargs, identifier? kwarg, - expr? kwargannotation, expr* defaults, - expr* kw_defaults) - arg = (identifier arg, expr? annotation) - - -- keyword arguments supplied to call - keyword = (identifier arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) -} diff --git a/docs/upgrade/ast/python3_3.ast b/docs/upgrade/ast/python3_3.ast deleted file mode 100644 index ad9e258..0000000 --- a/docs/upgrade/ast/python3_3.ast +++ /dev/null @@ -1,155 +0,0 @@ --- PYTHON 3.3 AST --- ASDL's five builtin types are identifier, int, string, bytes, object - -module Python version "3.3" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, - arguments args, - stmt* body, - expr* decorator_list, - expr? returns) - | ClassDef(identifier name, - expr* bases, - keyword* keywords, - expr? starargs, - expr? kwargs, - stmt* body, - expr* decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(withitem* items, stmt* body) - - | Raise(expr? exc, expr? cause) - | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier? module, alias* names, int? level) - - | Global(identifier* names) - | Nonlocal(identifier* names) - | Expr(expr value) - | Pass | Break | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Yield(expr? value) - | YieldFrom(expr value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords, - expr? starargs, expr? kwargs) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - | Bytes(bytes s) - | Ellipsis - -- other literals? bools? - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Starred(expr value, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And - | Or - - operator = Add - | Sub - | Mult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (arg* args, - identifier? vararg, - expr? varargannotation, - arg* kwonlyargs, - identifier? kwarg, - expr? kwargannotation, - expr* defaults, - expr* kw_defaults) - arg = (identifier arg, expr? annotation) - - -- keyword arguments supplied to call - keyword = (identifier arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) - - withitem = (expr context_expr, expr? optional_vars) -} diff --git a/docs/upgrade/ast/python3_4.ast b/docs/upgrade/ast/python3_4.ast deleted file mode 100644 index edfb756..0000000 --- a/docs/upgrade/ast/python3_4.ast +++ /dev/null @@ -1,156 +0,0 @@ --- Python 3.4 AST --- ASDL's six builtin types are identifier, int, string, bytes, object, singleton - -module Python version "3.4" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, - arguments args, - stmt* body, - expr* decorator_list, - expr? returns) - | ClassDef(identifier name, - expr* bases, - keyword* keywords, - expr? starargs, - expr? kwargs, - stmt* body, - expr* decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(withitem* items, stmt* body) - - | Raise(expr? exc, expr? cause) - | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier? module, alias* names, int? level) - - | Global(identifier* names) - | Nonlocal(identifier* names) - | Expr(expr value) - | Pass - | Break - | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Yield(expr? value) - | YieldFrom(expr value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords, - expr? starargs, expr? kwargs) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - | Bytes(bytes s) - | NameConstant(singleton value) - | Ellipsis - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Starred(expr value, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And - | Or - - operator = Add - | Sub - | Mult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - excepthandler = ExceptHandler(expr? type, - identifier? name, - stmt* body) - attributes (int lineno, - int col_offset) - - arguments = (arg* args, arg? vararg, arg* kwonlyargs, expr* kw_defaults, - arg? kwarg, expr* defaults) - - arg = (identifier arg, expr? annotation) - attributes (int lineno, int col_offset) - - -- keyword arguments supplied to call - keyword = (identifier arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) - - withitem = (expr context_expr, expr? optional_vars) -} diff --git a/docs/upgrade/ast/python3_5.ast b/docs/upgrade/ast/python3_5.ast deleted file mode 100644 index dfe5bf1..0000000 --- a/docs/upgrade/ast/python3_5.ast +++ /dev/null @@ -1,152 +0,0 @@ --- Python 3.5 AST --- ASDL's six builtin types are identifier, int, string, bytes, object, singleton - -module Python version "3.5" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, arguments args, - stmt* body, expr* decorator_list, expr? returns) - | AsyncFunctionDef(identifier name, arguments args, - stmt* body, expr* decorator_list, expr? returns) - - | ClassDef(identifier name, - expr* bases, - keyword* keywords, - stmt* body, - expr* decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | AsyncFor(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(withitem* items, stmt* body) - | AsyncWith(withitem* items, stmt* body) - - | Raise(expr? exc, expr? cause) - | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier? module, alias* names, int? level) - - | Global(identifier* names) - | Nonlocal(identifier* names) - | Expr(expr value) - | Pass | Break | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Await(expr value) - | Yield(expr? value) - | YieldFrom(expr value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - | Bytes(bytes s) - | NameConstant(singleton value) - | Ellipsis - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Starred(expr value, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And - | Or - - operator = Add - | Sub - | Mult - | MatMult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs) - - excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (arg* args, arg? vararg, arg* kwonlyargs, expr* kw_defaults, - arg? kwarg, expr* defaults) - - arg = (identifier arg, expr? annotation) - attributes (int lineno, int col_offset) - - -- keyword arguments supplied to call (NULL identifier for **kwargs) - keyword = (identifier? arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) - - withitem = (expr context_expr, expr? optional_vars) -} diff --git a/docs/upgrade/ast/python3_6.ast b/docs/upgrade/ast/python3_6.ast deleted file mode 100644 index 6cf106a..0000000 --- a/docs/upgrade/ast/python3_6.ast +++ /dev/null @@ -1,169 +0,0 @@ --- Python 3.6 AST --- ASDL's 7 builtin types are: --- identifier, int, string, bytes, object, singleton, constant --- --- singleton: None, True or False --- constant can be None, whereas None means "no value" for object. - -module Python version "3.6" -{ - mod = Module(stmt* body) - | Interactive(stmt* body) - | Expression(expr body) - - -- not really an actual node but useful in Jython's typesystem. - | Suite(stmt* body) - - stmt = FunctionDef(identifier name, - arguments args, - stmt* body, - expr* decorator_list, - expr? returns) - | AsyncFunctionDef(identifier name, - arguments args, - stmt* body, - expr* decorator_list, - expr? returns) - - | ClassDef(identifier name, - expr* bases, - keyword* keywords, - stmt* body, - expr* decorator_list) - | Return(expr? value) - - | Delete(expr* targets) - | Assign(expr* targets, expr value) - | AugAssign(expr target, operator op, expr value) - -- 'simple' indicates that we annotate simple name without parens - | AnnAssign(expr target, expr annotation, expr? value, int simple) - - -- use 'orelse' because else is a keyword in target languages - | For(expr target, expr iter, stmt* body, stmt* orelse) - | AsyncFor(expr target, expr iter, stmt* body, stmt* orelse) - | While(expr test, stmt* body, stmt* orelse) - | If(expr test, stmt* body, stmt* orelse) - | With(withitem* items, stmt* body) - | AsyncWith(withitem* items, stmt* body) - - | Raise(expr? exc, expr? cause) - | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) - | Assert(expr test, expr? msg) - - | Import(alias* names) - | ImportFrom(identifier? module, alias* names, int? level) - - | Global(identifier* names) - | Nonlocal(identifier* names) - | Expr(expr value) - | Pass - | Break - | Continue - - -- XXX Jython will be different - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - -- BoolOp() can use left & right? - expr = BoolOp(boolop op, expr* values) - | BinOp(expr left, operator op, expr right) - | UnaryOp(unaryop op, expr operand) - | Lambda(arguments args, expr body) - | IfExp(expr test, expr body, expr orelse) - | Dict(expr* keys, expr* values) - | Set(expr* elts) - | ListComp(expr elt, comprehension* generators) - | SetComp(expr elt, comprehension* generators) - | DictComp(expr key, expr value, comprehension* generators) - | GeneratorExp(expr elt, comprehension* generators) - -- the grammar constrains where yield expressions can occur - | Await(expr value) - | Yield(expr? value) - | YieldFrom(expr value) - -- need sequences for compare to distinguish between - -- x < 4 < 3 and (x < 4) < 3 - | Compare(expr left, cmpop* ops, expr* comparators) - | Call(expr func, expr* args, keyword* keywords) - | Num(object n) -- a number as a PyObject. - | Str(string s) -- need to specify raw, unicode, etc? - | FormattedValue(expr value, int? conversion, expr? format_spec) - | JoinedStr(expr* values) - | Bytes(bytes s) - | NameConstant(singleton value) - | Ellipsis - | Constant(constant value) - - -- the following expression can appear in assignment context - | Attribute(expr value, identifier attr, expr_context ctx) - | Subscript(expr value, slice slice, expr_context ctx) - | Starred(expr value, expr_context ctx) - | Name(identifier id, expr_context ctx) - | List(expr* elts, expr_context ctx) - | Tuple(expr* elts, expr_context ctx) - - -- col_offset is the byte offset in the utf8 string the parser uses - attributes (int lineno, int col_offset) - - expr_context = Load - | Store - | Del - | AugLoad - | AugStore - | Param - - slice = Slice(expr? lower, expr? upper, expr? step) - | ExtSlice(slice* dims) - | Index(expr value) - - boolop = And - | Or - - operator = Add - | Sub - | Mult - | MatMult - | Div - | Mod - | Pow - | LShift - | RShift - | BitOr - | BitXor - | BitAnd - | FloorDiv - - unaryop = Invert - | Not - | UAdd - | USub - - cmpop = Eq - | NotEq - | Lt - | LtE - | Gt - | GtE - | Is - | IsNot - | In - | NotIn - - comprehension = (expr target, expr iter, expr* ifs, int is_async) - - excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body) - attributes (int lineno, int col_offset) - - arguments = (arg* args, arg? vararg, arg* kwonlyargs, expr* kw_defaults, - arg? kwarg, expr* defaults) - - arg = (identifier arg, expr? annotation) - attributes (int lineno, int col_offset) - - -- keyword arguments supplied to call (NULL identifier for **kwargs) - keyword = (identifier? arg, expr value) - - -- import name with optional 'as' alias. - alias = (identifier name, identifier? asname) - - withitem = (expr context_expr, expr? optional_vars) -} diff --git a/docs/upgrade/index.rst b/docs/upgrade/index.rst deleted file mode 100644 index ebd8c57..0000000 --- a/docs/upgrade/index.rst +++ /dev/null @@ -1,83 +0,0 @@ -Concept for a upgrade to Python 3 -================================= - -RestrictedPython is a classic approach of compiler construction to create a limited subset of an existing programming language. - -Defining a programming language requires a regular grammar (`Chomsky 3 `_ / `EBNF `_) definition. -This grammar will be implemented in an abstract syntax tree (AST), which will be passed on to a code generator to produce a machine-readable version. - -.. _`_sec_code_generation`: - -Code generation ---------------- - -As Python is a platform independent programming language, this machine readable version is a byte code which will be translated on the fly by an interpreter into machine code. -This machine code then gets executed on the specific CPU architecture, with the standard operating system restrictions. - -The byte code produced must be compatible with the execution environment that the Python interpreter is running in, so we do not generate the byte code directly from ``compile_restricted`` and the other ``compile_restricted_*`` methods manually, it may not match what the interpreter expects. - -Thankfully, the Python ``compile()`` function introduced the capability to compile ``ast.AST`` code into byte code in Python 2.6, so we can return the platform-independent AST and keep byte code generation delegated to the interpreter. - -``compiler.ast`` --> ``ast`` ----------------------------- - -As the ``compiler`` module was deprecated in Python 2.6 and was removed before Python 3.0 was released it has never been available for any Python 3 version. -Instead Python 2.6 / Python 3 introduced the new ``ast`` module, that is more widly supported. -So we need to move from ``compiler.ast`` to ``ast`` to support newer Python versions. - -From the point of view of compiler design, the concepts of the ``compiler`` module and the ``ast`` module are similar. -The ``compiler`` module predates several major improvements of the Python development like a generally applied style guide. -While ``compiler`` still uses the old `CamelCase `_ Syntax (``visitNode(self, node, walker)``) the ``ast.AST`` did now use the Python common ``visit_Node(self, node)`` syntax. -Also the names of classes have been changed, where ``compiler`` uses ``Walker`` and ``Mutator`` the corresponding elements in ``ast.AST`` are ``NodeVisitor`` and ``NodeTransformator``. - -``ast`` module (Abstract Syntax Trees) --------------------------------------- - -The ``ast`` module consists of four areas: - -* ``AST`` (Basis of all Nodes) + all node class implementations -* ``NodeVisitor`` and ``NodeTransformer`` (tool to consume and modify the AST) -* Helper methods - - * ``parse`` - * ``walk`` - * ``dump`` - -* Constants - - * ``PyCF_ONLY_AST`` - -``NodeVisitor`` & ``NodeTransformer`` -..................................... - -A ``NodeVisitor`` is a class of a node / AST consumer, it reads the data by stepping through the tree without modifying it. -In contrast, a ``NodeTransformer`` (which inherits from a ``NodeVisitor``) is allowed to modify the tree and nodes. - -Modifying the AST ------------------ - -Technical Backgrounds - Links to External Documentation -....................................................... - -* Concept of Immutable Types and Python Example (https://en.wikipedia.org/wiki/Immutable_object#Python) -* Python 3 Standard Library Documentation on AST module ``ast`` (https://docs.python.org/3/library/ast.html) - - * AST Grammar of Python - - * `Python 3.7 AST `_ - * `Python 3.6 AST `_ - * `Python 3.5 AST `_ - * `Python 3.4 AST `_ - * `Python 3.3 AST `_ - * `Python 3.2 AST `_ - * `Python 3.1 AST `_ - * `Python 3.0 AST `_ - * `Python 2.7 AST `_ - * `Python 2.6 AST `_ - - * ``NodeVistiors`` (https://docs.python.org/3.5/library/ast.html#ast.NodeVisitor) - * ``NodeTransformer`` (https://docs.python.org/3.5/library/ast.html#ast.NodeTransformer) - * ``dump`` (https://docs.python.org/3.5/library/ast.html#ast.dump) - -* In detail Documentation on the Python AST module ``ast`` (Green Tree Snakes) (https://greentreesnakes.readthedocs.org/en/latest/) -* Example how to Instrumenting the Python AST (``ast.AST``) (http://www.dalkescientific.com/writings/diary/archive/2010/02/22/instrumenting_the_ast.html) diff --git a/docs/upgrade_dependencies/index.rst b/docs/upgrade_dependencies/index.rst index 62f02e8..55231a5 100644 --- a/docs/upgrade_dependencies/index.rst +++ b/docs/upgrade_dependencies/index.rst @@ -1,22 +1,9 @@ -Upgrade dependencies -==================== - -The following packages used in Zope2 and Plone depend on ``RestricedPython``: - -* AccessControl -* zope.untrustedpython -* DocumentTemplate -* Products.PageTemplates -* Products.PythonScripts -* Products.PluginIndexes -* five.pt (wrapping some functions and protection for Chameleon) - -Upgrade path ------------- +Upgrade from 3.x +---------------- For packages that use RestrictedPython the upgrade path differs on the actual usage. If it uses pure RestrictedPython without any additional checks it should be just to check the imports. -RestrictedPython did move some of the imports to the base namespace, so you should only import directly from ``RestrictedPython.__init__.py``. +RestrictedPython did move some of the imports to the base namespace, so you should only import directly from ``RestrictedPython``. * compile_restricted methods: diff --git a/docs/usage/api.rst b/docs/usage/api.rst index 32707c9..cd28565 100644 --- a/docs/usage/api.rst +++ b/docs/usage/api.rst @@ -1,87 +1,59 @@ API overview ------------ -RestrictedPython has tree major scopes: - -1. ``compile_restricted`` methods: +``compile_restricted`` methods +++++++++++++++++++++++++++++++ .. py:method:: compile_restricted(source, filename, mode, flags, dont_inherit, policy) :module: RestrictedPython Compiles source code into interpretable byte code. - :param source: (required). The source code that should be compiled - :param filename: (optional). - :param mode: (optional). - :param flags: (optional). - :param dont_inherit: (optional). - :param policy: (optional). - :type source: str or unicode text + :param source: (required). the source code that should be compiled + :param filename: (optional). defaults to ``''`` + :param mode: (optional). defaults to ``'exec'`` + :param flags: (optional). defaults to ``0`` + :param dont_inherit: (optional). defaults to ``False`` + :param policy: (optional). defaults to ``RestrictingNodeTransformer`` + :type source: str or unicode text or ``ast.AST`` :type filename: str or unicode text :type mode: str or unicode text :type flags: int :type dont_inherit: int :type policy: RestrictingNodeTransformer class - :return: Byte Code + :return: Python byte code .. py:method:: compile_restricted_exec(source, filename, flags, dont_inherit, policy) :module: RestrictedPython - Compiles source code into interpretable byte code. + Compiles source code into interpretable byte code with ``mode='exec'``. + The meaning and defaaults of the parameters are the same as in + ``compile_restricted``. - :param source: (required). The source code that should be compiled - :param filename: (optional). - :param flags: (optional). - :param dont_inherit: (optional). - :param policy: (optional). - :type source: str or unicode text - :type filename: str or unicode text - :type mode: str or unicode text - :type flags: int - :type dont_inherit: int - :type policy: RestrictingNodeTransformer class :return: CompileResult (a namedtuple with code, errors, warnings, used_names) .. py:method:: compile_restricted_eval(source, filename, flags, dont_inherit, policy) :module: RestrictedPython - Compiles source code into interpretable byte code. + Compiles source code into interpretable byte code with ``mode='eval'``. + The meaning and defaaults of the parameters are the same as in + ``compile_restricted``. - :param source: (required). The source code that should be compiled - :param filename: (optional). - :param flags: (optional). - :param dont_inherit: (optional). - :param policy: (optional). - :type source: str or unicode text - :type filename: str or unicode text - :type mode: str or unicode text - :type flags: int - :type dont_inherit: int - :type policy: RestrictingNodeTransformer class :return: CompileResult (a namedtuple with code, errors, warnings, used_names) .. py:method:: compile_restricted_single(source, filename, flags, dont_inherit, policy) :module: RestrictedPython - Compiles source code into interpretable byte code. + Compiles source code into interpretable byte code with ``mode='eval'``. + The meaning and defaaults of the parameters are the same as in + ``compile_restricted``. - :param source: (required). The source code that should be compiled - :param filename: (optional). - :param flags: (optional). - :param dont_inherit: (optional). - :param policy: (optional). - :type source: str or unicode text - :type filename: str or unicode text - :type mode: str or unicode text - :type flags: int - :type dont_inherit: int - :type policy: RestrictingNodeTransformer class :return: CompileResult (a namedtuple with code, errors, warnings, used_names) .. py:method:: compile_restricted_function(p, body, name, filename, globalize=None) :module: RestrictedPython - Compiles source code into interpretable byte code. + Compiles source code into interpretable byte code with ``mode='function'``. :param p: (required). :param body: (required). @@ -128,13 +100,30 @@ RestrictedPython has tree major scopes: ... compiled_function.__defaults__ or ()) >>> result = new_function(*[], **{}) -2. restricted builtins +restricted builtins ++++++++++++++++++++ * ``safe_globals`` * ``safe_builtins`` * ``limited_builtins`` * ``utility_builtins`` -3. helper modules +helper modules +++++++++++++++ * ``PrintCollector`` + + +RestrictingNodeTransformer +++++++++++++++++++++++++++ + +``RestrictingNodeTransformer`` provides the base policy used by RestrictedPython itself. + +It is a subclass of a ``NodeTransformer`` which has as set of ``visit_`` methods and a ``generic_visit`` method. + +``generic_visit`` is a predefined method of any ``NodeVisitor`` which sequentially visits all sub nodes. In RestrictedPython this behaviour is overwritten to always call a new internal method ``not_allowed(node)``. +This results in a implicit blacklisting of all not allowed AST elements. + +Any possibly new introduced AST element in Python (new language element) will implicitly be blocked and not allowed in RestrictedPython. + +So if new elements should be introduced an explicit ``visit_`` is necessary. diff --git a/docs/usage/basic_usage.rst b/docs/usage/basic_usage.rst index 27b21b9..076304f 100644 --- a/docs/usage/basic_usage.rst +++ b/docs/usage/basic_usage.rst @@ -108,6 +108,6 @@ To use ``for`` statements and comprehensions To use ``getattr`` you have to provide an implementation for it. - :func:``RestrictedPython.Guards.safer_getattr`` can be a starting point. + :func:`RestrictedPython.Guards.safer_getattr` can be a starting point. The usage of `RestrictedPython` in :mod:`AccessControl.ZopeGuards` can serve as example. diff --git a/docs/usage/framework_usage.rst b/docs/usage/framework_usage.rst index 6c78a34..0ff39a9 100644 --- a/docs/usage/framework_usage.rst +++ b/docs/usage/framework_usage.rst @@ -13,24 +13,21 @@ RestrictedPython provides four specialized compile_restricted methods: Those four methods return a named tuple (``CompileResult``) with four elements: -* ``code`` ```` object or ``None`` if ``errors`` is not empty -* ``errors`` a tuple with error messages -* ``warnings`` a list with warnings -* ``used_names`` a set / dictionary with collected used names of library calls +``code`` + ```` object or ``None`` if ``errors`` is not empty +``errors`` + a tuple with error messages +``warnings`` + a list with warnings +``used_names`` + a dictionary mapping collected used names to ``True``. -Those three information "lists" could be used to provide the user with informations about the compiled source code. +This information can be used to provide the user with informations about the compiled source code. -Typical uses cases for the four specialized methods: +Modifying the builtins is straight forward, it is just a dictionary containing the available library elements. +Modification normally means removing elements from existing builtins or adding allowed elements by copying from globals. -* ``compile_restricted_exec`` --> Python Modules or Scripts that should be used or called by the framework itself or from user calls -* ``compile_restricted_eval`` --> Templates -* ``compile_restricted_single`` -* ``compile_restricted_function`` - -Modifying the builtins is straight forward, it is just a dictionary containing access pointers to available library elements. -Modification is normally removing elements from existing builtins or adding allowed elements by copying from globals. - -For frameworks it could possibly also be useful to change handling of specific Python language elements. +For frameworks it could possibly also be useful to change the handling of specific Python language elements. For that use case RestrictedPython provides the possibility to pass an own policy. A policy is basically a special ``NodeTransformer`` that could be instantiated with three params for ``errors``, ``warnings`` and ``used_names``, it should be a subclass of RestrictedPython.RestrictingNodeTransformer. @@ -64,7 +61,7 @@ All ``compile_restricted*`` methods do have an optional parameter ``policy``, wh source_code, filename='', mode='exec', - policy=policy # Policy Class + policy=policy # policy class ) exec(byte_code, globals(), None) diff --git a/docs/usage/index.rst b/docs/usage/index.rst index 88b25c6..04b6186 100644 --- a/docs/usage/index.rst +++ b/docs/usage/index.rst @@ -1,7 +1,9 @@ Usage of RestrictedPython ========================= -.. include:: basic_usage.rst -.. include:: framework_usage.rst -.. include:: policy.rst -.. include:: api.rst +.. toctree:: + :maxdepth: 1 + + basic_usage + framework_usage + policy diff --git a/docs/usage/policy.rst b/docs/usage/policy.rst index 79d3cd4..8aabdcc 100644 --- a/docs/usage/policy.rst +++ b/docs/usage/policy.rst @@ -3,12 +3,7 @@ Policies & builtins ------------------- -.. todo:: - - Should be described in detail. - Especially the difference between builtins and a policy which is a NodeTransformer. - -RestrictedPython provides a way to define Policies, by redefining restricted versions of ``print``, ``getattr``, ``setattr``, ``import``, etc.. +RestrictedPython provides a way to define policies, by redefining restricted versions of ``print``, ``getattr``, ``setattr``, ``import``, etc.. As shortcuts it offers three stripped down versions of Pythons ``__builtins__``: .. _predefined_builtins: @@ -16,15 +11,16 @@ As shortcuts it offers three stripped down versions of Pythons ``__builtins__``: Predefined builtins ................... -.. todo:: +``safe_builtins`` + a safe set of builtin modules and functions +``limited_builtins`` + restricted sequence types (e. g. ``range``, ``list`` and ``tuple``) +``utility_builtins`` + access to standard modules like math, random, string and set. - Describe more in details - -* ``safe_builtins`` a safe set of builtin modules and functions, -* ``limited_builtins`` which provides restricted sequence types, -* ``utility_builtins`` which provides access for standard modules math, random, string and for sets. - -* ``safe_globals`` a shortcut for ``glb = {'__builtins__': safe_builtins}`` +``safe_globals`` is a shortcut for ``{'__builtins__': safe_builtins}`` as this +is the way globals have to be provided to the `exec` function to actually +restrict the access to the builtins provided by Python. Guards ...... @@ -46,3 +42,196 @@ Those and additional methods rely on a helper construct ``full_write_guard``, wh .. todo:: Describe full_write_guard more in detail and how it works. + +Implementing a policy +--------------------- + +RestrictedPython only provides the raw material for restricted execution. +To actually enforce any restrictions, you need to supply a policy +implementation by providing restricted versions of ``print``, +``getattr``, ``setattr``, ``import``, etc. These restricted +implementations are hooked up by providing a set of specially named +objects in the global dict that you use for execution of code. +Specifically: + +1. ``_print_`` is a callable object that returns a handler for print + statements. This handler must have a ``write()`` method that + accepts a single string argument, and must return a string when + called. ``RestrictedPython.PrintCollector.PrintCollector`` is a + suitable implementation. + +2. ``_write_`` is a guard function taking a single argument. If the + object passed to it may be written to, it should be returned, + otherwise the guard function should raise an exception. ``_write_`` + is typically called on an object before a ``setattr`` operation. + +3. ``_getattr_`` and ``_getitem_`` are guard functions, each of which + takes two arguments. The first is the base object to be accessed, + while the second is the attribute name or item index that will be + read. The guard function should return the attribute or subitem, + or raise an exception. + RestrictedPython ships with a default implementation + for ``_getattr_`` which prevents the following actions: + + * accessing an attribute those name start with an underscore + * accessing the format method of strings as it is considered harmful. + +4. ``__import__`` is the normal Python import hook, and should be used + to control access to Python packages and modules. + +5. ``__builtins__`` is the normal Python builtins dictionary, which + should be weeded down to a set that cannot be used to get around + your restrictions. A usable "safe" set is + ``RestrictedPython.Guards.safe_builtins``. + +To help illustrate how this works under the covers, here's an example +function: + +.. code-block:: python + + def f(x): + x.foo = x.foo + x[0] + print x + return printed + +and (sort of) how it looks after restricted compilation: + +.. code-block:: python + + def f(x): + # Make local variables from globals. + _print = _print_() + _write = _write_ + _getattr = _getattr_ + _getitem = _getitem_ + + # Translation of f(x) above + _write(x).foo = _getattr(x, 'foo') + _getitem(x, 0) + print >>_print, x + return _print() + +Examples +-------- + +``print`` +......... + +To support the ``print`` statement in restricted code, we supply a +``_print_`` object (note that it's a *factory*, e.g. a class or a +callable, from which the restricted machinery will create the object): + +.. code-block:: pycon + + >>> from RestrictedPython.PrintCollector import PrintCollector + >>> _print_ = PrintCollector + >>> _getattr_ = getattr + + >>> src = ''' + ... print("Hello World!") + ... ''' + >>> code = compile_restricted(src, '', 'exec') + >>> exec(code) + +As you can see, the text doesn't appear on stdout. The print +collector collects it. We can have access to the text using the +``printed`` variable, though: + +.. code-block:: pycon + + >>> src = ''' + ... print("Hello World!") + ... result = printed + ... ''' + >>> code = compile_restricted(src, '', 'exec') + >>> exec(code) + + >>> result + 'Hello World!\n' + +Built-ins +......... + +By supplying a different ``__builtins__`` dictionary, we can rule out +unsafe operations, such as opening files: + +.. code-block:: pycon + + >>> from RestrictedPython.Guards import safe_builtins + >>> restricted_globals = dict(__builtins__=safe_builtins) + + >>> src = ''' + ... open('/etc/passwd') + ... ''' + >>> code = compile_restricted(src, '', 'exec') + >>> exec(code, restricted_globals) + Traceback (most recent call last): + ... + NameError: name 'open' is not defined + +Guards +...... + +Here's an example of a write guard that never lets restricted code +modify (assign, delete an attribute or item) except dictionaries and +lists: + +.. code-block:: pycon + + >>> from RestrictedPython.Guards import full_write_guard + >>> _write_ = full_write_guard + >>> _getattr_ = getattr + + >>> class BikeShed(object): + ... colour = 'green' + ... + >>> shed = BikeShed() + +Normally accessing attriutes works as expected, because we're using +the standard ``getattr`` function for the ``_getattr_`` guard: + +.. code-block:: pycon + + >>> src = ''' + ... print(shed.colour) + ... result = printed + ... ''' + >>> code = compile_restricted(src, '', 'exec') + >>> exec(code) + + >>> result + 'green\n' + +However, changing an attribute doesn't work: + +.. code-block:: pycon + + >>> src = ''' + ... shed.colour = 'red' + ... ''' + >>> code = compile_restricted(src, '', 'exec') + >>> exec(code) + Traceback (most recent call last): + ... + TypeError: attribute-less object (assign or del) + +As said, this particular write guard (``full_write_guard``) will allow +restricted code to modify lists and dictionaries: + +.. code-block:: pycon + + >>> fibonacci = [1, 1, 2, 3, 4] + >>> transl = dict(one=1, two=2, tres=3) + >>> src = ''' + ... # correct mistake in list + ... fibonacci[-1] = 5 + ... # one item doesn't belong + ... del transl['tres'] + ... ''' + >>> code = compile_restricted(src, '', 'exec') + >>> exec(code) + + >>> fibonacci + [1, 1, 2, 3, 5] + + >>> sorted(transl.keys()) + ['one', 'two'] From d3bb99450433f8bb7c770e6a849fd1b415e727ac Mon Sep 17 00:00:00 2001 From: Michael Howitz Date: Thu, 9 May 2019 11:11:58 +0200 Subject: [PATCH 2/3] Apply suggestions from code review Co-Authored-By: Steffen Allner --- docs/idea.rst | 2 +- docs/upgrade_dependencies/index.rst | 2 +- docs/usage/api.rst | 12 ++++++------ docs/usage/framework_usage.rst | 2 +- docs/usage/policy.rst | 4 ++-- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/idea.rst b/docs/idea.rst index 8b9c8c3..9a7d5dc 100644 --- a/docs/idea.rst +++ b/docs/idea.rst @@ -48,7 +48,7 @@ For RestrictedPython this ``compile()`` method is replaced by: The primary parameter ``source`` has to be a string or ``ast.AST`` instance. Both methods either return compiled byte code that the interpreter can execute or raise exceptions if the provided source code is invalid. -As ``compile`` and ``compile_restricted`` just compile the provided source code to byte code it is not sufficient to sandbox the environment, as all calls to libraries are still available. +As ``compile`` and ``compile_restricted`` just compile the provided source code to byte code it is not sufficient as a sandbox environment, as all calls to libraries are still available. The two methods / statements: diff --git a/docs/upgrade_dependencies/index.rst b/docs/upgrade_dependencies/index.rst index 55231a5..4d295d1 100644 --- a/docs/upgrade_dependencies/index.rst +++ b/docs/upgrade_dependencies/index.rst @@ -2,7 +2,7 @@ Upgrade from 3.x ---------------- For packages that use RestrictedPython the upgrade path differs on the actual usage. -If it uses pure RestrictedPython without any additional checks it should be just to check the imports. +If it uses pure RestrictedPython without any additional checks it should be enough to check the imports. RestrictedPython did move some of the imports to the base namespace, so you should only import directly from ``RestrictedPython``. * compile_restricted methods: diff --git a/docs/usage/api.rst b/docs/usage/api.rst index cd28565..6d46360 100644 --- a/docs/usage/api.rst +++ b/docs/usage/api.rst @@ -27,7 +27,7 @@ API overview :module: RestrictedPython Compiles source code into interpretable byte code with ``mode='exec'``. - The meaning and defaaults of the parameters are the same as in + The meaning and defaults of the parameters are the same as in ``compile_restricted``. :return: CompileResult (a namedtuple with code, errors, warnings, used_names) @@ -36,7 +36,7 @@ API overview :module: RestrictedPython Compiles source code into interpretable byte code with ``mode='eval'``. - The meaning and defaaults of the parameters are the same as in + The meaning and defaults of the parameters are the same as in ``compile_restricted``. :return: CompileResult (a namedtuple with code, errors, warnings, used_names) @@ -45,7 +45,7 @@ API overview :module: RestrictedPython Compiles source code into interpretable byte code with ``mode='eval'``. - The meaning and defaaults of the parameters are the same as in + The meaning and defaults of the parameters are the same as in ``compile_restricted``. :return: CompileResult (a namedtuple with code, errors, warnings, used_names) @@ -119,11 +119,11 @@ RestrictingNodeTransformer ``RestrictingNodeTransformer`` provides the base policy used by RestrictedPython itself. -It is a subclass of a ``NodeTransformer`` which has as set of ``visit_`` methods and a ``generic_visit`` method. +It is a subclass of a ``NodeTransformer`` which has a set of ``visit_`` methods and a ``generic_visit`` method. ``generic_visit`` is a predefined method of any ``NodeVisitor`` which sequentially visits all sub nodes. In RestrictedPython this behaviour is overwritten to always call a new internal method ``not_allowed(node)``. -This results in a implicit blacklisting of all not allowed AST elements. +This results in an implicit blacklisting of all not allowed AST elements. Any possibly new introduced AST element in Python (new language element) will implicitly be blocked and not allowed in RestrictedPython. -So if new elements should be introduced an explicit ``visit_`` is necessary. +So, if new elements should be introduced, an explicit ``visit_`` is necessary. diff --git a/docs/usage/framework_usage.rst b/docs/usage/framework_usage.rst index 0ff39a9..f01f263 100644 --- a/docs/usage/framework_usage.rst +++ b/docs/usage/framework_usage.rst @@ -22,7 +22,7 @@ Those four methods return a named tuple (``CompileResult``) with four elements: ``used_names`` a dictionary mapping collected used names to ``True``. -This information can be used to provide the user with informations about the compiled source code. +These details can be used to inform the user about the compiled source code. Modifying the builtins is straight forward, it is just a dictionary containing the available library elements. Modification normally means removing elements from existing builtins or adding allowed elements by copying from globals. diff --git a/docs/usage/policy.rst b/docs/usage/policy.rst index 8aabdcc..80fae86 100644 --- a/docs/usage/policy.rst +++ b/docs/usage/policy.rst @@ -73,8 +73,8 @@ Specifically: RestrictedPython ships with a default implementation for ``_getattr_`` which prevents the following actions: - * accessing an attribute those name start with an underscore - * accessing the format method of strings as it is considered harmful. + * accessing an attribute whose name start with an underscore + * accessing the format method of strings as this is considered harmful. 4. ``__import__`` is the normal Python import hook, and should be used to control access to Python packages and modules. From c13c4f02f65b387c868b38cd1769def3dbb3ad64 Mon Sep 17 00:00:00 2001 From: Michael Howitz Date: Thu, 9 May 2019 11:12:27 +0200 Subject: [PATCH 3/3] Update docs/usage/api.rst --- docs/usage/api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/usage/api.rst b/docs/usage/api.rst index 6d46360..1cc8645 100644 --- a/docs/usage/api.rst +++ b/docs/usage/api.rst @@ -21,7 +21,7 @@ API overview :type flags: int :type dont_inherit: int :type policy: RestrictingNodeTransformer class - :return: Python byte code + :return: Python ``code`` object .. py:method:: compile_restricted_exec(source, filename, flags, dont_inherit, policy) :module: RestrictedPython