Skip to content

Commit

Permalink
Improve results returned by estimation functions (#343)
Browse files Browse the repository at this point in the history
  • Loading branch information
@janosg
janosg authored Jun 7, 2022
1 parent b69b16e commit 8076105
Show file tree
Hide file tree
Showing 20 changed files with 2,125 additions and 515 deletions.
12 changes: 11 additions & 1 deletion CHANGES.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ releases are available on `Anaconda.org
0.3.0
-----

Fist release with pytree support in optimization, estimation and differentiation.
Fist release with pytree support in optimization, estimation and differentiation
and much better result objects in optimization and estimation.

Breaking changes

Expand All @@ -27,6 +28,15 @@ Breaking changes
- Convenience functions to create namedtuples are removed from ``estimagic.utilities``.


- :gh:`345` Moves estimation_table to new latex functionality of pandas
(:ghuser:`mpetrosian`)
- :gh:`344` Adds pytree support to slice_plot (:ghuser:`janosg`)
- :gh:`343` Improves the result object of estimation functions and makes msm estimation
pytree compatible (:ghuser:`janosg`)
- :gh:`342` Improves default options of the fides optimizer, allows single constraints
and polishes the documentation (:ghuser:`janosg`)
- :gh:`340` Enables history collection for optimizers that evaluate the criterion
function in parallel (:ghuser:`janosg`)
- :gh:`339` Incorporates user feedback and polishes the documentation.
- :gh:`338` Improves log reading functions (:ghuser:`janosg`)
- :gh:`336` Adds pytree support to the dashboard (:ghuser:`roecla`).
Expand Down
2 changes: 1 addition & 1 deletion codecov.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ coverage:
status:
patch:
default:
target: 85%
target: 80%
project:
default:
target: 90%
Expand Down
202 changes: 190 additions & 12 deletions docs/source/getting_started/estimation/first_likelihood_estimation_with_estimagic.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@
}
],
"source": [
"res[\"summary_jacobian\"].round(3)"
"res.summary().round(3)"
]
},
{
Expand All @@ -287,39 +287,217 @@
"source": [
"## What's in the results?\n",
"\n",
"The result of `estimate_ml` is a dictionary with the following entries:"
"`LikelihoodResult` objects provide attributes and methods to calculate standard errors, confidence intervals and p_values with multiple methods. You can even calculate cluster robust standard errors. \n",
"\n",
"A few examples are:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>value</th>\n",
" <th>lower_bound</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>intercept</th>\n",
" <td>2.075153</td>\n",
" <td>-inf</td>\n",
" </tr>\n",
" <tr>\n",
" <th>slope</th>\n",
" <td>-0.885302</td>\n",
" <td>-inf</td>\n",
" </tr>\n",
" <tr>\n",
" <th>sd</th>\n",
" <td>1.028189</td>\n",
" <td>1.000000e-10</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
"dict_keys(['summary_jacobian', 'cov_jacobian', 'jacobian', 'hessian', 'optimize_res', 'jacobian_numdiff_info'])"
" value lower_bound\n",
"intercept 2.075153 -inf\n",
"slope -0.885302 -inf\n",
"sd 1.028189 1.000000e-10"
]
},
"execution_count": 8,
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"res.keys()"
"res.params"
]
},
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>intercept</th>\n",
" <th>slope</th>\n",
" <th>sd</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>intercept</th>\n",
" <td>0.010575</td>\n",
" <td>-0.000296</td>\n",
" <td>0.001165</td>\n",
" </tr>\n",
" <tr>\n",
" <th>slope</th>\n",
" <td>-0.000296</td>\n",
" <td>0.009050</td>\n",
" <td>-0.000952</td>\n",
" </tr>\n",
" <tr>\n",
" <th>sd</th>\n",
" <td>0.001165</td>\n",
" <td>-0.000952</td>\n",
" <td>0.003336</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" intercept slope sd\n",
"intercept 0.010575 -0.000296 0.001165\n",
"slope -0.000296 0.009050 -0.000952\n",
"sd 0.001165 -0.000952 0.003336"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Importantly, we might have several summaries and several cov entries. This is because we always all possible types of standard errors, so you can compare them easily (for example using our estimation table functions). \n",
"\n",
"In the current example we only have jacobian based standard errors because we did not provide a closed form hessian function and numerical hessians are not yet implemented. With a closed form hessian we would also get hessian based and robust standard errors. Even cluster and strata robust standard errors are possible if you provide the relevant information. \n",
"\n",
"If numerical optimizations or derivative calculations were performed, the full output of those steps is also part of the results dictionary. "
"res.cov(method=\"robust\")"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>value</th>\n",
" <th>lower_bound</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>intercept</th>\n",
" <td>0.105445</td>\n",
" <td>-inf</td>\n",
" </tr>\n",
" <tr>\n",
" <th>slope</th>\n",
" <td>0.111271</td>\n",
" <td>-inf</td>\n",
" </tr>\n",
" <tr>\n",
" <th>sd</th>\n",
" <td>0.094721</td>\n",
" <td>1.000000e-10</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" value lower_bound\n",
"intercept 0.105445 -inf\n",
"slope 0.111271 -inf\n",
"sd 0.094721 1.000000e-10"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"res.se()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
Expand Down
Loading

0 comments on commit 8076105

Please sign in to comment.